openstack/swift - swift - OpenDev: Free Software Needs Free Tools

In swift conf files, boolean options use different
format: some use true/false, and some use True/False.
This patch is aim to using lowcase true/false to unify
boolean params formats in swift conf files.
Fix Bug #1203421
Change-Id: I3e1bfc6e43231f51e0710aa54869f3774ee896b1

If the cursors are not closed, then when
create_function is called, if they are not
GC'd then create_function will fail. On Pythons
without reference counting (e.g. PyPy) they
will not be GC'd immediately.
Change-Id: I39210616d323691ccb745149f24430a7a61382ec

Using a list comprehension here makes the code less
readable, and is also slower (since it has to allocate
and grow a list that is unused).
Change-Id: I74cc9d2cf2e271a2c45d45cc5ea777189c02eacb

A failure to close files in a timely fassion means
that data is not necessarily written immediately
on Pythons which do not use reference counting
(e.g. PyPy).
Change-Id: I5d363249676032a025a22a67275c2eed3151b264

* new module swift.obj.diskfile
I parameterized two constants from obj.server into the DiskFile's __init__
 * DATADIR -> obj_dir
 * DISALLOWED_HEADERS -> disallowed_metadata_keys
I'm not sure if this is the right long term abstraction but for now it avoids
circular imports.
Change-Id: I3962202c07c4b2fbfc26f9776c8a5c96292ae199

Refactor the various auditors to rely on the audit_location_generator
yielding tuples containing paths with the expected suffix.
We also fix the exception handling for container_sync to not expect a
broker object (since the act of creating a broker object can raise an
exception).
For the object auditor we removed an unneeded check for disk_file
since get_data_file_size() will raise DiskFileNotExist under the same
condition (raises code coverage slightly).
Change-Id: I11d405e629063177ef21543b75e9076da1a03b61

The unit tests were playing fast and loose with the tombstone marker,
where the test framework was setting up a DiskFile object which had
its data written to the .ts file, not the .data file. This behavior
did not reflect how the interfaces to DiskFile were supposed to
work.
Change-Id: Idd6e8882e062ba2e13489f14189223ab4158677c
Signed-off-by: Peter Portante <peter.portante@redhat.com>

- Fixes bug 1201431.
Change-Id: If025a41caf3a629b9efb4d67c53c423796d37a91

Instead of blacklisting Hacking globally,
only blacklist those that currently occur frequently
(for a later followup patch), and fix the rest. In
detail:
H101 Use TODO(NAME)
H201 no 'except:' at least use 'except Exception:'
H231 octal number 022 should be written as 0o22
H401 docstring should not start with a space
H701 Empty localization string
Change-Id: Ib3b3d56b68d1cf15d3b67ac9749fcbdb876dc52a

Remove logic of allowed methods list from object, container and account
servers. Instead of it add replicator decorator to utils and use new
decorator for REPLICATE methods in object/account/container servers.
This decorator mark method as special for usfor use only by the
replication.
If the option replication_server is not used, then this mechanism is not
enabled. If the replicaton_server option is set (not None) then the
respective server is a replicator (option value is True) and should use
ONLY the methods marked for replication server using the decorator, or
it is a normal server type and should NOT use methods marked for the
replication server.
Change-Id: I1041b31413cd0c39000317cc57a8c27816e1dfe8

Change-Id: I19ced30b6e92133766310f8e41286202607c5b4c

- Automatically add underscore if there wasn't one already.
- This makes consistent with tempauth.
- Fixes bug 1198878.
- This should probably need to be mentioned in the release changelog for
 upgrades.
Change-Id: Ia82f14f82c380c7ed2ddd190116f1493e410cfbe

test_DELETE_x_container_headers_with_more_container_replicas() would
sometimes fail, and it's because a test helper method was defaulting
to sorting the captured headers by X-Container-Partition. Well, in
tests we use FakeRing, and the partition is always 1, so that sorting
didn't buy us much. Now the sorting is done by X-Container-Device,
which actually differs meaningfully.
Change-Id: Ibe5b2fcd3f23280ed2caaa703111a98861331866

This reverts commit 68cb91097b
Just so this is not get forgotten in the tree...
We are using daemon mode and chunked is not supported in this mode.

Change-Id: I043a80c38091f59ce6707730363a4b43b29ae6ec

* Create class for testing _repl_to_not and replicate_object fuctions to
 prevent duplication code by adding all preparation into setUp function.
* Move existed test function which testin _repl_to_not and
 replicate_object into created classes.
* Add tests for replicate_object and _repl_to_node functions.
Change-Id: I75ac7c6f0230e71bfb24328e44c33734b520b4cd

The behavior's the same as before; this just makes the code (IMHO) a
little easier to read.
Change-Id: Ie54d836d81af7410057219f60b72b840c9ce51b9

Change-Id: I5d6d6d6c32b6573474288897f6fa174b6f150183

Change-Id: Ic2c79d52770208c22b39cb2b00f2f8a6a86e8aa7

In past couple of years, the XFS team has greatly improved inode use in
xfs. With more recent kernels, there is no performance penalty for
using the default inode size, and a smaller inode size gives us
improvements in other areas where disk access is involved.
DocImpact
Change-Id: Ie9da53a6e8bf43d1d02881befbb52595462c9f2e

As reported in the documentation bug, the apache deployment guide's
reference to apache2 mod_wsgi not supporting client chunked encoding
has become outdated. It now supports this feature, using an optional
parameter.
Updated the paragraph in question to reflect this
patchset 2 mentions the WSGIChunkedRequest variable and adds it
to the sample configs - On by default. Feedback welcome
fixes bug 1194935
Change-Id: I07c5c8506ac34e1e0e08fa6d961babde2f9b7367

Change-Id: Id03f130ad7016b11b740050b7aadf879773d5a5f

Formpost wouldn't look at X-Account-Meta-Temp-URL-Key-2 for signature
validation, which was confusing to users. Now it does, and there's a
utility function that formpost and tempurl both use to help prevent
divergence in the future.
Also removed formpost's custom cache-lookup logic. Formpost was
looking for tempurl keys in memcache, but it's been broken since
commit 21343ab (2013年05月02日) when tempurl changed its memcache keys, so
the lookup would always miss. (Since tempurl started storing *lists*
of keys, formpost would have choked on the new data, so that oversight
was the only thing keeping formpost working at all.)
Now formpost just uses the same get_account_info() call as everyone
else does, so it gets all the benefits of the new caching framework.
Change-Id: I426d8f2882ebe3ce373de604b9ff18846f5999e3

We've gone back and forth about this. In the initial commit, it couldn't
possibly work because you wouldn't be able to get the Etags to match. Then it
was expressly disallowed with a custom error message, and now its allowed. The
reason we're allowing it is that 1,000 segments isn't enough for some use cases
and we decided its better than just upping the number of allowed segments. The
code to make it work isn't all that complicated and it allows for virtually
unlimited SLO object size. There is also a new configurable limit on the
maximum connection time for both SLOs and DLOs defaulting to 1 day. This will
hopefully alleviate worries about infinite requests. Think I'll leave the
python-swift client support for nested SLOs to somebody else though :).
DocImpact
Change-Id: Id16187481b37e716d2bd09bdbab8cc87537e3ddd

Making this smaller (10 instead of 18) can make some of the tests run
faster and makes rebuilding of the rings faster.
Change-Id: Ibe46011d8e6a6482d39b3a20ac9c091d9fbc6ef7

When checking referer (.r, .rlistings), tempauth return 401/403 if '.r'
exists without '.rlistings' or obj. Actually it should skip this kind of
authorizing and check the following authorizing.
And remove some unnecessary lines
Change-Id: Ib60e59b7c80331a699cb5c3496f948cf32135d0b

Dead since e499b91.
Change-Id: I3c571ae471eb4a0b95772b19eac90183cfc7fe22

Change-Id: If04c88f7e686f4b4709c5b68351ebbbea97d12bd

Currently clients can not distinguish between non existing accounts
(which can be created) and accounts marked for deletion, which has
not yet been reaped and therefore cannot be re-created until reaped.
Following this patch, if an account is marked as deleted but hasn't
been reaped and is still on disk, responses will include a status
header:
 'X-Account-Status' = 'Deleted'
Fixes:Bug #1188609
Change-Id: Ibd39965ae3f5d45fd78f130e0e31f5a0141a8633

Change-Id: I61ffbaba01aba86d61e4f3d988b5abeee8f07813