Add fallocate_reserve to account and container servers.

Code Issues Proposed changes

The object server can be configured to leave a certain amount of disk
space free; default is 1%. This is useful in avoiding 100%-full
filesystems, as those can get Swift in a state where the filesystem is
too full to write tombstones, so you can't delete objects to free up
space.
When a cluster has accounts/containers and objects on the same disks,
then you can wind up with a 100%-full disk since account and container
servers don't respect fallocate_reserve. This commit makes account and
container servers respect fallocate_reserve so that disks shared
between account/container and object rings won't get 100% full.
When a disk's free space falls below the configured reserve, account
and container PUT, POST, and REPLICATE requests will fail with a 507
status code. These are the operations that can significantly increase
the disk space used by a given database.
I called the parameter "fallocate_reserve" for consistency with the
object server. No actual fallocate() call happens under Swift's
control in the account or container servers (sqlite3 might make such a
call, but it's out of our hands).
Change-Id: I083442eef14bf83c0ea717b1decb3e6b56dbf1d0

This commit is contained in:

Samuel Merritt

2018年02月05日 14:06:18 -08:00

committed by Matthew Oliver

parent bea74a3260

commit 8e651a2d3d

10 changed files with 321 additions and 7 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

25
doc/source/admin_guide.rst

View File

@@ -298,10 +298,27 @@ Preventing Disk Full Scenarios

Prevent disk full scenarios by ensuring that the ``proxy-server`` blocks PUT

requests and rsync prevents replication to the specific drives.

You can prevent `proxy-server` PUT requests to low space disks by ensuring

``fallocate_reserve`` is set in the ``object-server.conf``. By default,

``fallocate_reserve`` is set to 1%. This blocks PUT requests that leave the

free disk space below 1% of the disk.

You can prevent `proxy-server` PUT requests to low space disks by

ensuring ``fallocate_reserve`` is set in ``account-server.conf``,

``container-server.conf``, and ``object-server.conf``. By default,

``fallocate_reserve`` is set to 1%. In the object server, this blocks

PUT requests that would leave the free disk space below 1% of the

disk. In the account and container servers, this blocks operations

that will increase account or container database size once the free

disk space falls below 1%.

Setting ``fallocate_reserve`` is highly recommended to avoid filling

disks to 100%. When Swift's disks are completely full, all requests

involving those disks will fail, including DELETE requests that would

otherwise free up space. This is because object deletion includes the

creation of a zero-byte tombstone (.ts) to record the time of the

deletion for replication purposes; this happens prior to deletion of

the object's data. On a completely-full filesystem, that zero-byte .ts

file cannot be created, so the DELETE request will fail and the disk

will remain completely full. If ``fallocate_reserve`` is set, then the

filesystem will have enough space to create the zero-byte .ts file,

and thus the deletion of the object will succeed and free up some

space.

In order to prevent rsync replication to specific drives, firstly

setup ``rsync_module`` per disk in your ``object-replicator``.

Add fallocate_reserve to account and container servers.

25 doc/source/admin_guide.rst Unescape Escape View File

25
doc/source/admin_guide.rst

View File