openstack/swift - swift - OpenDev: Free Software Needs Free Tools

The current behavior is really painful when you've got hundreds of shard
ranges in a DB. The new summary with the states is default. Users can
add a -v/--verbose flag to see the old full detail view.
Change-Id: I0a7d65f64540f99514c52a70f9157ef060a8a892

Hopefully this will fix the currently-broken probe test gate?
Depends-On: https://review.opendev.org/#/c/736070/
Change-Id: Ib652534b35236fdb6bcab131c7dc08a079bf72f6

When you've got a container server that's struggling to respond to HEAD
requests, it's really not helpful to keep evicting the cache when some
of the hundreds of concurrent requests trying to repopulate the cache
return 503.
Change-Id: I49174a21a854a4e8e564a7bbf997e1841f9dda71
Closes-Bug: #1883211 

Closes-Bug: #1883561
Change-Id: Idbc13b90a565c4e64cef9035dbcd60872576727a

https://docs.aws.amazon.com/cli/latest/userguide/cliv2-migration.html#cliv2-migration-s3-copy-metadata
AWS CLI version 2 improves Amazon S3 handling of file properties
and tags when performing multipart copies. We still don't supprt
object tagging hence the aws s3 cp command fails for mulitpart
copies with default options.
This way get tagging request will receive an empty tagset in
response and mulitpart copies will work fine
Change-Id: I1f031b05025cafac00e86966c240aa5f7258d0bf

crudini seems to have trouble on py3 -- still not sure *why* it's using
py3 for the losf job, though...
Change-Id: Id98055994c8d59e561372417c9eb4aec969afc6a

We previously fixed a bunch of places, but not quite *all* the places;
at the very least, some account-layer services (like the replicator and
auditor IIRC) could still bomb out -- and it's important that
replicators still respect fallocate_reserve!
Now, do the NicerInterpolation thing every time we call readconf.
Additionally, clean up the original fix to avoid globally
monkey-patching configparser.
Related-Bug: #1844368
Closes-Bug: #1872553
Change-Id: I4512e686cde37930f0482909f537220a57fef76b

This way proxies log memcached errors in the normal way instead of
to the root logger (which eventually gets them out on STDERR).
If no logger is provided, fall back to the root logger behavior.
Change-Id: I2f7b3e7d5b976fab07c9a2d0a9b8c0bd9a840dfd

Now that we can have null bytes in Swift paths, we need a way for
operators to be able to locate such containers and objects. Our usual
trick of making sure the name is properly quoted for the shell won't
suffice; running something like
 swift-get-nodes /etc/swift/container.ring.gz $'AUTH_test/0円versions0円container'
has the path get cut off after "AUTH_test/" because of how argv works.
So, add a new option, --quoted, to let operators indicate that they
already quoted the path.
Drive-bys:
 * If account, container, or object are explicitly blank, treat them
 as though they were not provided. This provides better errors when
 account is explicitly blank, for example.
 * If account, container, or object are not provided or explicitly
 blank, skip printing them. This resolves abiguities about things
 like objects whose name is actually "None".
 * When displaying account, container, and object, quote them (since
 they may contain newlines or other control characters).
Change-Id: I3d10e121b403de7533cc3671604bcbdecb02c795
Related-Change: If912f71d8b0d03369680374e8233da85d8d38f85
Closes-Bug: #1875734
Closes-Bug: #1875735
Closes-Bug: #1875736
Related-Bug: #1791302 

This was released in 2.24.0, which already has a release note for it.
Change-Id: I9837df281ec8baa19e8e4a7976f415e8add4a2da

We've been seeing more TIMEOUT failures lately where the jobs still seem
to be making steady (if slow) progress.
Change-Id: I19c1f48bace551c78ad0c6c8b6ccad75e44e8904

This traceback would probably show up in the wild, but it's not relevant
to this test.
Change-Id: I9e6e7679f674bddddcc4440b38d5741aaece3393

...unless the client requests it specifically using a new flag:
 X-Backend-Auto-Create: true
Previously, you could get real jittery listings during a rebalance:
 * Partition with a shard DB get reassigned, so one primary has no DB.
 * Proxy makes a listing, gets a 404, tries another node. Likely, one of
 the other shard replicas responds. Things are fine.
 * Update comes in. Since we use the auto_create_account_prefix
 namespace for shards, container DB gets created and we write the row.
 * Proxy makes another listing. There's a one-in-three chance that we
 claim there's only one object in that whole range.
Note that unsharded databases would respond to the update with a 404 and
wait for one of the other primaries (or the old primary that's now a
hand-off) to rsync a whole DB over, keeping us in the happy state.
Now, if the account is in the shards namespace, 404 the object update if
we have no DB. Wait for replication like in the unsharded case.
Continue to be willing to create the DB when the sharder is seeding all
the CREATED databases before it starts cleaving, though.
Change-Id: I15052f3f17999e6f432951ba7c0731dcdc9475bb
Closes-Bug: #1881210 

Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems
Update Sphinx version as well.
Disable openstackdocs_auto_name to use 'project' variable as name.
Set openstackdocs_pdf_link to link to PDF file. Note that
the link to the published document only works on docs.openstack.org
where the PDF file is placed in the top-level html directory. The
site-preview places the PDF in a pdf directory.
Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.
Remove docs requirements from lower-constraints, they are not needed
during install or test but only for docs building.
openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.
See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html
Change-Id: I131850d2a5c6164dfd48c9c95885d4754b5236c6

The requirements repo is support python 3.5 as oldest python version
while swift still supports py27.
thus, requirements-check will fail on a couple of lines in swift. The
check is only run when these files are touched.
The py2.7 packagers we know about aren't depending on upstream
requirements.txt for correctness and aside from all the production
deployments running on py2.7 we only realistically support >=py3.7
There's no good reason for our requirements.txt to be "unspported" by
the openstack requirements check job. Since they only support >=py3.5
we can change our requirements.txt inline with that. This should be
fine for everything we could hope to get out of both our
requirements.txt and the check!
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: Ibf8000498528c401707be8b0b91b8355cd993786

Change-Id: I227e8c2defe96fb5f33054167677ae551830d9e8

This patch removed the separate s3api, staticweb functional tests
gate jobs and added them across all other functional test jobs.
Change-Id: Ie1c606132a054defc2b3cc14a66031090e7b8449

Also, run the in-process encryption func tests like that.
Change-Id: I984ab8d1304d23b89589973950b10dda4aea0db3

Change-Id: I9218aaeb5fcd21f1bc2a5d655e3216059a209aeb

The idea is, if none of
 - timestamp,
 - object_count,
 - bytes_used,
 - state, or
 - epoch
has changed, we shouldn't need to send an update back to the root
container.
This is more-or-less comparable to what the container-updater does to
avoid unnecessary writes to the account.
Closes-Bug: #1834097
Change-Id: I1ee7ba5eae3c508064714c4deb4f7c6bbbfa32af

Change-Id: I5b0ae0b78d37a31928e0b34a1b6118802a3a8236

Change-Id: I175b9e1464494454b1193b9b9d5311cb6fd6503f

Following https://github.com/openstack/devstack/commit/6b6bdc711,
the swift-dsvm-functional job has been running everything under
Python 3 anyway.
Change-Id: Ie285f513e1ed71dbaf4e12fe747ea6087664f843