openstack/devstack-plugin-ceph - devstack-plugin-ceph - OpenDev: Free Software Needs Free Tools

We were using Jewel/Luminous. Being a developer tool, we should
be sticking with newer versions.
This patch set also adds the following changes
- Refactor setup_packages_for_manila_on_<distro> functions
- Removes ceph_init call. Seems it lost relevance
- Removes APT repositories setup. We directly use shaman/chacra
packages directly
- Removes RUN_AS env var. It has been a while since Ceph
defaults to ceph user to run all the services. No longer needed.
- Removes sleep calls for services start checks. We use systemctl
checks.
- Adds more debugging logs
- Uses the search API from ceph shaman instead of hardwiring urls
- Adds the jq (JSON processor) dependency
Change-Id: I51c90e592070b99422e692d5e9e97083d93146e8

QA people pointed out that It should be set here, not in the job.
Change-Id: Id925fe667472f7a71ac236aec136dd7b59e43794

I5698e451861828a8b1240d046d1610d8d37ca5a2 introduced support for this
test in Stein so enable the compute feature from Stein onwards.
Blueprint: extend-in-use-rbd-volumes
Change-Id: I6b2575b12fdb9636f106b3ca28681e29ad3eb8ac

Related-Bug: #1653953
Depends-On: https://review.opendev.org/#/c/668123/
Change-Id: I3e1f9683a28a2e2e67a0327031f0670f33391126

Related-Bug: #1653953
Depends-On: https://review.opendev.org/#/c/668119/
Change-Id: I1aa87fc16a37b6051e55c35ae69ef911a87d4c11

Related-Bug: #1653953
Depends-On: https://review.opendev.org/668118/
Change-Id: I132e26486782918d31a9ac10332f2802342883a9

The dependent nova fix resolves the issue with rbd and
shelve/unshelve testing in tempest. Re-enabling the test
needs to be branch-specific since devstack-plugin-ceph
is branch-less but the fixes in nova will be branch
specific so we don't want to enable this on any nova
branch that does not have the fix since it would break
the job on those branches.
Related-Bug: #1653953
Depends-On: https://review.opendev.org/457886/
Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>
Change-Id: Ifbfc15033020c430a93574f2726b2279c9be130d

Similar to what happened some time ago for the stable/queens branch, we
are now seeing issues again on master after there was a new Paste
release being made a couple of weeks ago. So we need to perform
the same workaround of forcing pip to install PasteDeploy over
the distro version on the master branch.
Change-Id: I3e98a08e78182bdf5ec09b30e31b98435ec29456
Closes-Bug: 1803143

The workaround in order to fix the issue with Paste/PasteDeploy being
pulled in as system packages when installing ceph is only needed for the
Pike UCA. Assume that we only use that when devstack is running at the
stable/queens branch.
Change-Id: I0bbf04be015b777b3feb2c5e837b1f46f81cf3e9

This fixes a small logical error, we only want to set a libvirt secret
if nova is enabled, not both cinder and nova. This prevents a
deployment with cinder but not nova from being successful.
Change-Id: Ief83fe661cf1b2099af8131a7a206ffca3458dc7

The gate-tempest-dsvm-full-devstack-plugin-ceph-ubuntu-xenial job is
currently failing at a rate of 100% since we updated to use the Pike
UCA. The Ceph packages in the Pike UCA are pulling in python-paste and
python-pastedeploy. When devstack pip installs a newer version of
python-paste to satisfy upper-constraints, we end up with Paste
installed from pypi and python-pastedeploy installed from the Ubuntu
archive and Keystone fails to start with the error:
 "ImportError: cannot import name deploy."
This adds a hack to force an update of PasteDeploy from pypi during the
devstack "stack install" phase to work around the problem and get the
job passing again.
Change-Id: Idf28de8faa56c2d387291c7d14ac6d4bf1c74c6a

This is to workaround this test which is not currently
working:
https://review.openstack.org/#/c/471352/
Change-Id: I9506e326f9432eda3a057996805d4b0cd7a91446

Ceph doesn't support volume swapping so disable that feature in
tempest.
Change-Id: I8f3da1d730113b100526662ba7ec71f603ab36da

This commit introduces the support of deploying a containerized ceph
cluster. It relies on the ceph/demo container image available as part of
the ceph-docker project (https://github.com/ceph/ceph-docker).
To enable this scenario, just set CEPH_CONTAINERIZED to True, all the
previous available options will work. So settings like
CEPH_LOOPBACK_DISK_SIZE continue to work.
Change-Id: Ie10155140448c04b88b9920381b54505f7359585
Signed-off-by: Sébastien Han <seb@redhat.com>

This patch enables the plugin to setup ceph radosgw and configures
it as a swift-compatible endpoint with keystone integration.
ENABLE_CEPH_RGW is the new variable introduced, which is False
by default, and can be set to True in localrc if radosgw needs
to be setup.
Fixed couple of other (related) things ...
1) Created rgw specific functions
2) Checking for radosgw & swift co-existence, erroring out early enough
NOTE: Tested on ubuntu trusty only
TODOs ...
1) Not tested on Fedora (F22), radosgw startup has some issues there.
Co-Authored-By: Deepak C Shetty <deepakcs@redhat.com>
Change-Id: I21db4168eb69d107599f6b6ab1668b02b764b2c6

The cleanup_ceph_embedded function nukes the ceph
conf directories, so when subsequently trying
to stop services they aren't stopped. Next stack.sh
run will fail because daemons are still running.
Fix by stopping daemons before running cleanup.
Change-Id: I51cd58eae83171a47aa898b65240b72ea0217af9
Closes-Bug: #1534600 

Let the plugin setup and configure CephFS as the storage backend for
Manila. This is not done by default. Refer the README to do so.
Add 'pre_test_hook' and 'post_test_hook' scripts that would be needed
to configure Manila-Ceph CI jobs.
The following changes are also made:
* Install a development version of Ceph that is compatible with Manila's
 CephFSNative driver.
* The development version is a version greater than Infernalis. And
 from Infernalis onwards, the ceph daemons are run as user "ceph".
 So allow the daemons to run as user "ceph" for such versions.
* Enhance get_ceph_version to check Ceph version even when the
 ceph mon daemon is not up, by checking the CLI version.
Co-Authored-By: John Spray <john.spray@redhat.com>
Change-Id: I74314bfcc6b52d524bb84f2232a988f275b9afbf

Add configurables so that one need not have Ceph be the storage
backend for Cinder, Cinder-backup, Glance and Nova.
Let Ceph be the default storage backend for Cinder, Cinder backup,
Glance, and Nova. Allow the default setting to be overriden when the
global config, ENABLE_CEPH_{CINDER,C_BAK,GLANCE,NOVA} is set to
False in the local.conf/localrc file.
Closes Bug: #1521278
Change-Id: I3afe7c57d5d56f15fe533ad5b1dbe36ccd369e6b

Co-Authored-By: Deepak C Shetty <deepakcs@redhat.com>
Change-Id: I124b54b5068fbcd46e278281da12ea27f0034d1f