Files
e8affa7db5817e50cd82130a8f4c250725b80884
swift /setup.cfg

156 lines
5.9 KiB
INI
Raw Normal View History

Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
[metadata]
name = swift
summary = OpenStack Object Storage
description_file =
README.rst
long_description_content_type = text/x-rst
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
author = OpenStack
author_email = openstack-discuss@lists.openstack.org
url = https://docs.openstack.org/swift/latest/
project_urls =
Documentation = https://docs.openstack.org/swift/latest/
Bug Tracker = https://bugs.launchpad.net/swift
Source Code = https://opendev.org/openstack/swift/
Release Notes = https://opendev.org/openstack/swift/src/branch/master/CHANGELOG
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
classifier =
Development Status :: 5 - Production/Stable
Environment :: OpenStack
Intended Audience :: Information Technology
Intended Audience :: System Administrators
License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 2.7
Programming Language :: Python :: 3
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9
Programming Language :: Python :: 3.10
Programming Language :: Python :: 3.11
Programming Language :: Python :: 3.12
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
[pbr]
skip_authors = True
skip_changelog = True
skip_reno = True
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
[options]
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
packages =
swift
[files]
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
scripts =
bin/swift-account-audit
bin/swift-account-auditor
bin/swift-account-info
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
bin/swift-account-reaper
bin/swift-account-replicator
bin/swift-config
bin/swift-container-auditor
bin/swift-container-info
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
bin/swift-container-replicator
Add sharder daemon, manage_shard_ranges tool and probe tests The sharder daemon visits container dbs and when necessary executes the sharding workflow on the db. The workflow is, in overview: - perform an audit of the container for sharding purposes. - move any misplaced objects that do not belong in the container to their correct shard. - move shard ranges from FOUND state to CREATED state by creating shard containers. - move shard ranges from CREATED to CLEAVED state by cleaving objects to shard dbs and replicating those dbs. By default this is done in batches of 2 shard ranges per visit. Additionally, when the auto_shard option is True (NOT yet recommeneded in production), the sharder will identify shard ranges for containers that have exceeded the threshold for sharding, and will also manage the sharding and shrinking of shard containers. The manage_shard_ranges tool provides a means to manually identify shard ranges and merge them to a container in order to trigger sharding. This is currently the recommended way to shard a container. Co-Authored-By: Alistair Coles <alistairncoles@gmail.com> Co-Authored-By: Tim Burke <tim.burke@gmail.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Change-Id: I7f192209d4d5580f5a0aa6838f9f04e436cf6b1f
2018年05月02日 10:47:51 +01:00
bin/swift-container-sharder
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
bin/swift-container-sync
bin/swift-container-updater
bin/swift-container-reconciler
bin/swift-reconciler-enqueue
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
bin/swift-dispersion-populate
bin/swift-dispersion-report
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
bin/swift-drive-audit
bin/swift-form-signature
bin/swift-get-nodes
bin/swift-init
bin/swift-object-auditor
bin/swift-object-expirer
bin/swift-object-info
bin/swift-object-replicator
Erasure Code Reconstructor This patch adds the erasure code reconstructor. It follows the design of the replicator but: - There is no notion of update() or update_deleted(). - There is a single job processor - Jobs are processed partition by partition. - At the end of processing a rebalanced or handoff partition, the reconstructor will remove successfully reverted objects if any. And various ssync changes such as the addition of reconstruct_fa() function called from ssync_sender which performs the actual reconstruction while sending the object to the receiver Co-Authored-By: Alistair Coles <alistair.coles@hp.com> Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: John Dickinson <me@not.mn> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Co-Authored-By: Tushar Gohad <tushar.gohad@intel.com> Co-Authored-By: Samuel Merritt <sam@swiftstack.com> Co-Authored-By: Christian Schwede <christian.schwede@enovance.com> Co-Authored-By: Yuan Zhou <yuan.zhou@intel.com> blueprint ec-reconstructor Change-Id: I7d15620dc66ee646b223bb9fff700796cd6bef51
2014年10月28日 09:51:06 -07:00
bin/swift-object-reconstructor
Add support to increase object ring partition power This patch adds methods to increase the partition power of an existing object ring without downtime for the users using a 3-step process. Data won't be moved to other nodes; objects using the new increased partition power will be located on the same device and are hardlinked to avoid data movement. 1. A new setting "next_part_power" will be added to the rings, and once the proxy server reloaded the rings it will send this value to the object servers on any write operation. Object servers will now create a hard-link in the new location to the original DiskFile object. Already existing data will be relinked using a new tool in the new locations using hardlinks. 2. The actual partition power itself will be increased. Servers will now use the new partition power to read from and write to. No longer required hard links in the old object location have to be removed now by the relinker tool; the relinker tool reads the next_part_power setting to find object locations that need to be cleaned up. 3. The "next_part_power" flag will be removed. This mostly implements the spec in [1]; however it's not using an "epoch" as described there. The idea of the epoch was to store data using different partition powers in their own namespace to avoid conflicts with auditors and replicators as well as being able to abort such an operation and just remove the new tree. This would require some heavy change of the on-disk data layout, and other object-server implementations would be required to adopt this scheme too. Instead the object-replicator is now aware that there is a partition power increase in progress and will skip replication of data in that storage policy; the relinker tool should be simply run and afterwards the partition power will be increased. This shouldn't take that much time (it's only walking the filesystem and hardlinking); impact should be low therefore. The relinker should be run on all storage nodes at the same time in parallel to decrease the required time (though this is not mandatory). Failures during relinking should not affect cluster operations - relinking can be even aborted manually and restarted later. Auditors are not quarantining objects written to a path with a different partition power and therefore working as before (though they are reading each object twice in the worst case before the no longer needed hard links are removed). Co-Authored-By: Alistair Coles <alistair.coles@hpe.com> Co-Authored-By: Matthew Oliver <matt@oliver.net.au> Co-Authored-By: Tim Burke <tim.burke@gmail.com> [1] https://specs.openstack.org/openstack/swift-specs/specs/in_progress/ increasing_partition_power.html Change-Id: I7d6371a04f5c1c4adbb8733a71f3c177ee5448bb
2016年07月04日 18:21:54 +02:00
bin/swift-object-relinker
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
bin/swift-object-updater
bin/swift-oldies
bin/swift-orphans
bin/swift-recon
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
bin/swift-recon-cron
bin/swift-ring-builder
Add ring-builder analyzer. This is a tool to help developers quantify changes to the ring builder. It takes a scenario (JSON file) describing the builder's basic parameters (part_power, replicas, etc.) and a number of "rounds", where each round is a set of operations to perform on the builder. For each round, the operations are applied, and then the builder is rebalanced until it reaches a steady state. The idea is that a developer observes the ring builder behaving suboptimally, writes a scenario to reproduce the behavior, modifies the ring builder to fix it, and references the scenario with the commit so that others can see that things have improved. I decided to write this after writing my fourth or fifth hacky one-off script to reproduce some bad behavior in the ring builder. Change-Id: I114242748368f142304aab90a6d99c1337bced4c
2015年06月11日 15:40:28 -07:00
bin/swift-ring-builder-analyzer
bin/swift-ring-composer
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
[extras]
kms_keymaster =
oslo.config>=4.0.0,!=4.3.0,!=4.4.0 # Apache-2.0
castellan>=0.13.0 # Apache-2.0
kmip_keymaster =
pykmip>=0.7.0 # Apache-2.0
keystone =
keystonemiddleware>=4.17.0
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
[entry_points]
Add sharder daemon, manage_shard_ranges tool and probe tests The sharder daemon visits container dbs and when necessary executes the sharding workflow on the db. The workflow is, in overview: - perform an audit of the container for sharding purposes. - move any misplaced objects that do not belong in the container to their correct shard. - move shard ranges from FOUND state to CREATED state by creating shard containers. - move shard ranges from CREATED to CLEAVED state by cleaving objects to shard dbs and replicating those dbs. By default this is done in batches of 2 shard ranges per visit. Additionally, when the auto_shard option is True (NOT yet recommeneded in production), the sharder will identify shard ranges for containers that have exceeded the threshold for sharding, and will also manage the sharding and shrinking of shard containers. The manage_shard_ranges tool provides a means to manually identify shard ranges and merge them to a container in order to trigger sharding. This is currently the recommended way to shard a container. Co-Authored-By: Alistair Coles <alistairncoles@gmail.com> Co-Authored-By: Tim Burke <tim.burke@gmail.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Change-Id: I7f192209d4d5580f5a0aa6838f9f04e436cf6b1f
2018年05月02日 10:47:51 +01:00
console_scripts =
Use entry_points for server executables The old [files]scripts method of specifying executable Python scripts triggers some legacy easy-install-like mode for editable installs, which relies on pkg_resources. Recent versions of setuptools (67.5.0+) have started emitting warnings when importing pkg_resources, which in turn cause quite noticeable slowdowns in process startup. This is particularly prominant on py312, which stopped pre-installing (an often older version of) setuptools in new venvs. See also: - https://github.com/python/cpython/issues/95299 - https://github.com/pypa/setuptools/pull/3843 - https://github.com/pypa/setuptools/issues/3966 Now, use [entry_points]console_scripts to specify these executables, which does not use pkg_resources in the generated script files. Change-Id: Ifcc8138e7b55d5b82bea0d411ec6bfcca2c77c83
2024年05月06日 15:10:26 -07:00
swift-account-server = swift.account.server:main
Add operator tool to async-delete some or all objects in a container Adds a tool, swift-container-deleter, that takes an account/container and optional prefix, marker, and/or end-marker; spins up an internal client; makes listing requests against the container; and pushes the found objects into the object-expirer queue with a special application/async-deleted content-type. In order to do this enqueuing efficiently, a new internal-to-the-cluster container method is introduced: UPDATE. It takes a JSON list of object entries and runs them through merge_items. The object-expirer is updated to look for work items with this content-type and skip the X-If-Deleted-At check that it would normally do. Note that the target-container's listing will continue to show the objects until data is actually deleted, bypassing some of the concerns raised in the related change about clearing out a container entirely and then deleting it. Change-Id: Ia13ee5da3d1b5c536eccaadc7a6fdcd997374443 Related-Change: I50e403dee75585fc1ff2bb385d6b2d2f13653cf8
2019年03月27日 15:28:50 -07:00
swift-container-deleter = swift.cli.container_deleter:main
Use entry_points for server executables The old [files]scripts method of specifying executable Python scripts triggers some legacy easy-install-like mode for editable installs, which relies on pkg_resources. Recent versions of setuptools (67.5.0+) have started emitting warnings when importing pkg_resources, which in turn cause quite noticeable slowdowns in process startup. This is particularly prominant on py312, which stopped pre-installing (an often older version of) setuptools in new venvs. See also: - https://github.com/python/cpython/issues/95299 - https://github.com/pypa/setuptools/pull/3843 - https://github.com/pypa/setuptools/issues/3966 Now, use [entry_points]console_scripts to specify these executables, which does not use pkg_resources in the generated script files. Change-Id: Ifcc8138e7b55d5b82bea0d411ec6bfcca2c77c83
2024年05月06日 15:10:26 -07:00
swift-container-server = swift.container.server:main
swift-manage-shard-ranges = swift.cli.manage_shard_ranges:main
swift-object-server = swift.obj.server:main
swift-proxy-server = swift.proxy.server:main
Add a swift-reload command Previously, WSGI server systemd unit files might have used something like ExecReload=kill -USR1 $MAINPID This was risky; in the related change, reloads were made safer, but required more than one ExecReload line. Meanwhile, systemd docs (https://www.freedesktop.org/software/systemd/man/systemd.service.html#ExecReload=) say > It is strongly recommended to set ExecReload= to a command that > not only triggers a configuration reload of the daemon, but also > synchronously waits for it to complete. which *neither* set of ExecReloads would do. Now, add a new swift-reload command which, given a pid, * validates that the PID seems to belong to a Swift WSGI server manager process, * checks that the config used by that PID is still valid, * signals the PID to perform a seamless reload, and * waits for the reload to complete by monitoring the PID's children. As a result, WSGI server systemd unit files can now use something like ExecReload=swift-reload $MAINPID to follow systemd recommendations. Change-Id: Ifcadd2f8427f107aae1921cdd311f7973b0312e1 Related-Change: I9e5e158ce8be92535430b9cabf040063f5188bf4
2022年03月10日 11:44:39 -08:00
swift-reload = swift.cli.reload:main
Add sharder daemon, manage_shard_ranges tool and probe tests The sharder daemon visits container dbs and when necessary executes the sharding workflow on the db. The workflow is, in overview: - perform an audit of the container for sharding purposes. - move any misplaced objects that do not belong in the container to their correct shard. - move shard ranges from FOUND state to CREATED state by creating shard containers. - move shard ranges from CREATED to CLEAVED state by cleaving objects to shard dbs and replicating those dbs. By default this is done in batches of 2 shard ranges per visit. Additionally, when the auto_shard option is True (NOT yet recommeneded in production), the sharder will identify shard ranges for containers that have exceeded the threshold for sharding, and will also manage the sharding and shrinking of shard containers. The manage_shard_ranges tool provides a means to manually identify shard ranges and merge them to a container in order to trigger sharding. This is currently the recommended way to shard a container. Co-Authored-By: Alistair Coles <alistairncoles@gmail.com> Co-Authored-By: Tim Burke <tim.burke@gmail.com> Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Change-Id: I7f192209d4d5580f5a0aa6838f9f04e436cf6b1f
2018年05月02日 10:47:51 +01:00
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
paste.app_factory =
proxy = swift.proxy.server:app_factory
object = swift.obj.server:app_factory
mem_object = swift.obj.mem_server:app_factory
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
container = swift.container.server:app_factory
account = swift.account.server:app_factory
paste.filter_factory =
healthcheck = swift.common.middleware.healthcheck:filter_factory
crossdomain = swift.common.middleware.crossdomain:filter_factory
memcache = swift.common.middleware.memcache:filter_factory
read_only = swift.common.middleware.read_only:filter_factory
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
ratelimit = swift.common.middleware.ratelimit:filter_factory
2022年03月31日 16:36:32 +01:00
backend_ratelimit = swift.common.middleware.backend_ratelimit:filter_factory
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
cname_lookup = swift.common.middleware.cname_lookup:filter_factory
catch_errors = swift.common.middleware.catch_errors:filter_factory
domain_remap = swift.common.middleware.domain_remap:filter_factory
staticweb = swift.common.middleware.staticweb:filter_factory
tempauth = swift.common.middleware.tempauth:filter_factory
keystoneauth = swift.common.middleware.keystoneauth:filter_factory
recon = swift.common.middleware.recon:filter_factory
tempurl = swift.common.middleware.tempurl:filter_factory
formpost = swift.common.middleware.formpost:filter_factory
name_check = swift.common.middleware.name_check:filter_factory
bulk = swift.common.middleware.bulk:filter_factory
container_quotas = swift.common.middleware.container_quotas:filter_factory
account_quotas = swift.common.middleware.account_quotas:filter_factory
proxy_logging = swift.common.middleware.proxy_logging:filter_factory
Move all DLO functionality to middleware This is for the same reason that SLO got pulled into middleware, which includes stuff like automatic retry of GETs on broken connection and the multi-ring storage policy stuff. The proxy will automatically insert the dlo middleware at an appropriate place in the pipeline the same way it does with the gatekeeper middleware. Clusters will still support DLOs after upgrade even with an old config file that doesn't mention dlo at all. Includes support for reading config values from the proxy server's config section so that upgraded clusters continue to work as before. Bonus fix: resolve 'after' vs. 'after_fn' in proxy's required filters list. Having two was confusing, so I kept the more-general one. DocImpact blueprint multi-ring-large-objects Change-Id: Ib3b3830c246816dd549fc74be98b4bc651e7bace
2013年11月21日 17:31:16 -08:00
dlo = swift.common.middleware.dlo:filter_factory
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
slo = swift.common.middleware.slo:filter_factory
list_endpoints = swift.common.middleware.list_endpoints:filter_factory
Generic means for persisting system metadata. Middleware or core features may need to store metadata against accounts or containers. This patch adds a generic mechanism for system metadata to be persisted in backend databases, without polluting the user metadata namespace, by using the reserved header namespace x-<server_type>-sysmeta-*. Modifications are firstly that backend servers persist system metadata headers alongside user metadata and other system state. For accounts and containers, system metadata in PUT and POST requests is treated in a similar way to user metadata. System metadata is not yet supported for object requests. Secondly, changes in the proxy controllers ensure that headers in the system metadata namespace will pass through in requests to backend servers. Thirdly, system metadata returned from backend servers in GET or HEAD responses is added to the cached info dict, which middleware can access. Finally, a gatekeeper middleware module is provided which filters all system metadata headers from requests and responses by removing headers with names starting x-account-sysmeta-, x-container-sysmeta-. The gatekeeper also removes headers starting x-object-sysmeta- in anticipation of future support for system metadata being set for objects. This prevents clients from writing or reading system metadata. The required_filters list in swift/proxy/server.py is modified to include the gatekeeper middleware so that if the gatekeeper has not been configured in the pipeline then it will be automatically inserted close to the start of the pipeline. blueprint cluster-federation Change-Id: I80b8b14243cc59505f8c584920f8f527646b5f45
2013年12月03日 22:02:39 +00:00
gatekeeper = swift.common.middleware.gatekeeper:filter_factory
New container sync configuration option Summary of the new configuration option: The cluster operators add the container_sync middleware to their proxy pipeline and create a container-sync-realms.conf for their cluster and copy this out to all their proxy and container servers. This file specifies the available container sync "realms". A container sync realm is a group of clusters with a shared key that have agreed to provide container syncing to one another. The end user can then set the X-Container-Sync-To value on a container to //realm/cluster/account/container instead of the previously required URL. The allowed hosts list is not used with this configuration and instead every container sync request sent is signed using the realm key and user key. This offers better security as source hosts can be faked much more easily than faking per request signatures. Replaying signed requests, assuming it could easily be done, shouldn't be an issue as the X-Timestamp is part of the signature and so would just short-circuit as already current or as superceded. This also makes configuration easier for the end user, especially with difficult networking situations where a different host might need to be used for the container sync daemon since it's connecting from within a cluster. With this new configuration option, the end user just specifies the realm and cluster names and that is resolved to the proper endpoint configured by the operator. If the operator changes their configuration (key or endpoint), the end user does not need to change theirs. DocImpact Change-Id: Ie1704990b66d0434e4991e26ed1da8b08cb05a37
2013年12月08日 09:13:59 +00:00
container_sync = swift.common.middleware.container_sync:filter_factory
xprofile = swift.common.middleware.xprofile:filter_factory
versioned_writes = swift.common.middleware.versioned_writes:filter_factory
Refactor server side copy as middleware Rewrite server side copy and 'object post as copy' feature as middleware to simplify the PUT method in the object controller code. COPY is no longer a verb implemented as public method in Proxy application. The server side copy middleware is inserted to the left of dlo, slo and versioned_writes middlewares in the proxy server pipeline. As a result, dlo and slo copy_hooks are no longer required. SLO manifests are now validated when copied so when copying a manifest to another account the referenced segments must be readable in that account for the manifest copy to succeed (previously this validation was not made, meaning the manifest was copied but could be unusable if the segments were not readable). With this change, there should be no change in functionality or existing behavior. This is asserted with (almost) no changes required to existing functional tests. Some notes (for operators): * Middleware required to be auto-inserted before slo and dlo and versioned_writes * Turning off server side copy is not configurable. * object_post_as_copy is no longer a configurable option of proxy server but of this middleware. However, for smooth upgrade, config option set in proxy server app is also read. DocImpact: Introducing server side copy as middleware Co-Authored-By: Alistair Coles <alistair.coles@hpe.com> Co-Authored-By: Thiago da Silva <thiago@redhat.com> Change-Id: Ic96a92e938589a2f6add35a40741fd062f1c29eb Signed-off-by: Prashanth Pai <ppai@redhat.com> Signed-off-by: Thiago da Silva <thiago@redhat.com>
2015年02月18日 11:59:31 +05:30
copy = swift.common.middleware.copy:filter_factory
Enable object body and metadata encryption Adds encryption middlewares. All object servers and proxy servers should be upgraded before introducing encryption middleware. Encryption middleware should be first introduced with the encryption middleware disable_encryption option set to True. Once all proxies have encryption middleware installed this option may be set to False (the default). Increases constraints.py:MAX_HEADER_COUNT by 4 to allow for headers generated by encryption-related middleware. Co-Authored-By: Tim Burke <tim.burke@gmail.com> Co-Authored-By: Christian Cachin <cca@zurich.ibm.com> Co-Authored-By: Mahati Chamarthy <mahati.chamarthy@gmail.com> Co-Authored-By: Peter Chng <pchng@ca.ibm.com> Co-Authored-By: Alistair Coles <alistair.coles@hpe.com> Co-Authored-By: Jonathan Hinson <jlhinson@us.ibm.com> Co-Authored-By: Hamdi Roumani <roumani@ca.ibm.com> UpgradeImpact Change-Id: Ie6db22697ceb1021baaa6bddcf8e41ae3acb5376
2016年06月07日 15:01:32 +01:00
keymaster = swift.common.middleware.crypto.keymaster:filter_factory
encryption = swift.common.middleware.crypto:filter_factory
kms_keymaster = swift.common.middleware.crypto.kms_keymaster:filter_factory
kmip_keymaster = swift.common.middleware.crypto.kmip_keymaster:filter_factory
listing_formats = swift.common.middleware.listing_formats:filter_factory
Symlink implementation. Add a symbolic link ("symlink") object support to Swift. This object will reference another object. GET and HEAD requests for a symlink object will operate on the referenced object. DELETE and PUT requests for a symlink object will operate on the symlink object, not the referenced object, and will delete or overwrite it, respectively. POST requests are *not* forwarded to the referenced object and should be sent directly. POST requests sent to a symlink object will result in a 307 Error. Historical information on symlink design can be found here: https://github.com/openstack/swift-specs/blob/master/specs/in_progress/symlinks.rst. https://etherpad.openstack.org/p/swift_symlinks Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: Janie Richling <jrichli@us.ibm.com> Co-Authored-By: Kazuhiro MIYAHARA <miyahara.kazuhiro@lab.ntt.co.jp> Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp> Change-Id: I838ed71bacb3e33916db8dd42c7880d5bb9f8e18 Signed-off-by: Thiago da Silva <thiago@redhat.com>
2015年10月07日 15:14:58 -04:00
symlink = swift.common.middleware.symlink:filter_factory
Import swift3 into swift repo as s3api middleware This attempts to import openstack/swift3 package into swift upstream repository, namespace. This is almost simple porting except following items. 1. Rename swift3 namespace to swift.common.middleware.s3api 1.1 Rename also some conflicted class names (e.g. Request/Response) 2. Port unittests to test/unit/s3api dir to be able to run on the gate. 3. Port functests to test/functional/s3api and setup in-process testing 4. Port docs to doc dir, then address the namespace change. 5. Use get_logger() instead of global logger instance 6. Avoid global conf instance Ex. fix various minor issue on those steps (e.g. packages, dependencies, deprecated things) The details and patch references in the work on feature/s3api are listed at https://trello.com/b/ZloaZ23t/s3api (completed board) Note that, because this is just a porting, no new feature is developed since the last swift3 release, and in the future work, Swift upstream may continue to work on remaining items for further improvements and the best compatibility of Amazon S3. Please read the new docs for your deployment and keep track to know what would be changed in the future releases. Change-Id: Ib803ea89cfee9a53c429606149159dd136c036fd Co-Authored-By: Thiago da Silva <thiago@redhat.com> Co-Authored-By: Tim Burke <tim.burke@gmail.com>
2017年10月16日 21:39:12 +09:00
s3api = swift.common.middleware.s3api.s3api:filter_factory
s3token = swift.common.middleware.s3api.s3token:filter_factory
Middleware that allows a user to have quoted Etags Users have complained for a while that Swift's ETags don't match the expected RFC formats. We've resisted fixing this for just as long, worrying that the fix would break innumerable clients that expect the value to be a hex-encoded MD5 digest and *nothing else*. But, users keep asking for it, and some consumers (including some CDNs) break if we *don't* have quoted etags -- so, let's make it an option. With this middleware, Swift users can set metadata per-account or even per-container to explicitly request RFC compliant etags or not. Swift operators also get an option to change the default behavior cluster-wide; it defaults to the old, non-compliant format. See also: - https://tools.ietf.org/html/rfc2616#section-3.11 - https://tools.ietf.org/html/rfc7232#section-2.3 Closes-Bug: 1099087 Closes-Bug: 1424614 Co-Authored-By: Tim Burke <tim.burke@gmail.com> Change-Id: I380c6e34949d857158e11eb428b3eda9975d855d
2019年12月19日 14:24:51 -05:00
etag_quoter = swift.common.middleware.etag_quoter:filter_factory
Migrate to pbr for build pbr is the libification of what was openstack.common.setup. If provides the build information in a delcarative form, instead of as executable python code, which works around the chicken and egg problem of needing setup libraries present to run setup, but needing to run setup to tell if you need setup libraries. One of the features that comes along with this is versioning based on git tags. If the current revision is a signed git tag, then that is the version of the package. If it is not, the version is equal to the most recent git tag, plus a commit count, plus a git sha (similar to git describe, but scrubbed for python version rules compliance) pbr updates are also part of the upcoming automation around ensuring global requirements stay in sync. Closes-Bug: #1179007 Change-Id: Ia473960be7e8aa44f09d48cea72ed3c8845f82fa
2013年05月11日 15:35:37 -04:00
2018年06月29日 16:13:05 +02:00
swift.diskfile =
replication.fs = swift.obj.diskfile:DiskFileManager
erasure_coding.fs = swift.obj.diskfile:ECDiskFileManager
2010年07月19日 11:22:40 -05:00
Let developers/operators add watchers to object audit Swift operators may find it useful to operate on each object in their cluster in some way. This commit provides them a way to hook into the object auditor with a simple, clearly-defined boundary so that they can iterate over their objects without additional disk IO. For example, a cluster operator may want to ensure a semantic consistency with all SLO segments accounted in their manifests, or locate objects that aren't in container listings. Now that Swift has encryption support, this could be used to locate unencrypted objects. The list goes on. This commit makes the auditor locate, via entry points, the watchers named in its config file. A watcher is a class with at least these four methods: __init__(self, conf, logger, **kwargs) start(self, audit_type, **kwargs) see_object(self, object_metadata, data_file_path, **kwargs) end(self, **kwargs) The auditor will call watcher.start(audit_type) at the start of an audit pass, watcher.see_object(...) for each object audited, and watcher.end() at the end of an audit pass. All method arguments are passed as keyword args. This version of the API is implemented on the context of the auditor itself, without spawning any additional processes. If the plugins are not working well -- hang, crash, or leak -- it's easier to debug them when there's no additional complication of processes that run by themselves. In addition, we include a reference implementation of plugin for the watcher API, as a help to plugin writers. Change-Id: I1be1faec53b2cdfaabf927598f1460e23c206b0a
2015年08月13日 17:05:25 -05:00
swift.object_audit_watcher =
dark_data = swift.obj.watchers.dark_data:DarkDataWatcher
[bdist_wheel]
universal = 1