openstack/devstack-plugin-ceph - devstack-plugin-ceph - OpenDev: Free Software Needs Free Tools

The CEPH_DATA_DIR variable holds the location of Ceph drives and
objects. It defaults to '/var/lib/ceph'.
Replace hardcoded occurrences of '/var/lib/ceph' with
'${CEPH_DATA_DIR}'.
Change-Id: I2570e37f9135d853854600acfbfff3b768e4f0e3

By default bashate checks for lines longer than 80
characters and issues E006 warnings for these.
Suppress the E006 warnings since we don't enforce
this line length standard in OpenStack shell
scripts so that other warnings will be more evident.
Change-Id: Icdf32cb7e3d810d2f8644674d3516ed808c873a5

Restacking with ceph plugin fails when trying to install
the cinder user's ceph authorization key in libvirt's list
of secrets because there is already a secret in that list
with the same uuid. The function _undefine_virsh_secret
is run on cleanup and should have removed that key from
the libvirt secret list.
Fix a typo in the variable name for the uuid that identifies
the authorization key in the secret list so that the
cleanup routine removes the key as intended.
Closes-Bug: #1570665
Change-Id: I19d2c20979dca49d73c427c5b1413229ff39f394

The rule_id line is formatted as:
 "rule_id": 0,
Which means the id integer is the second column of the parsed output,
not the third.
Since a third column doesn't exist, the call to cut returns nothing,
leaving $RULE_ID invalid, which then causes the subsequent ruleset
commands to fail.
We don't see this normally because $CEPH_REPLICAS is set to 1 by
default.
Change-Id: I0cc57f5abc023cb8b2faeea940f28ab54cd290ff
Related-Bug: #1581159 

Hammer LTS 0.94.6 is currently available for Fedora 22 and 23 [1].
Fedora 21 is EOL and should be removed from the list [2].
[1] https://admin.fedoraproject.org/pkgdb/package/rpms/ceph/
[2] https://fedoraproject.org/wiki/Releases
Change-Id: I5fc7f25ed145c29cadbe03d945511aaa12c54310

In commit bb773e7285 the key for
verifying release packages from ceph.com was removed when support for
the ceph package mirrors was added. However, it is still needed for
non-gate case of fetching packages from ceph.com, so this adds it
back in that case.
Closes-Bug: #1578494
Change-Id: I18f8927f42710e9365c246c55f26f53403965ee1

For the ceph job to be voting, infra requires that we pull packages from
a pre-configured repository that's mirrored and reliable for gate jobs.
They have arranged for ceph.com packages to be mirrored and will store a
link to those packages in /etc/apt/sources.list.d/ceph.list. If this
file is present, we should use it. If not, then we're free to update
the sources list to fetch the latest packages.
This approach is required to regain voting status in the gate.
Change-Id: I5f811c917e6964c33440ed644597c9e485963626

The changes in Ceph that were needed to make manila's cephfs_native
driver work are available in the Jewel branch. It's no longer
necessary to use the 'wip-manila' branch.
Also, with Jewel branch, manila's libcephfs client needs to be root
to perform certain manila operations. So set manila's client as root.
Change-Id: I715f79a6801b12522c047b1df4d950c99a1ffad5

The cephfs_native driver CI does not need the service image that is
being uploaded. Disable the upload until a need arises, for example,
when the CI does scenario testing.
Change-Id: I1ed0af5fd61829a075a41df2939f0377deb351b3

This patch elaborates on fine-grained control of each service by listing
examples instead of using pseudocode - which could be confusing to
newcomers.
Change-Id: I9ba3c7b4460aa840e88c2c8c82aa70ba06c8302c

If we want to put the disk image into a tmpfs, changing CEPH_DATA_DIR
won't work because some file operation needed by ceph are not allowed
on tmpfs.
So, this change allows to change the disk image path only instead of the
whole CEPH_DATA_DIR.
Change-Id: If92f0bdca16f866a475820cf71a30ded28fb2572

So if the directory exists the stack does not fail.
Change-Id: Ibdd2883b5328d58d7c2a74d97dfe2e0ea87d65ee
Signed-off-by: Sébastien Han <seb@redhat.com>

Since we're about to be voting and I expect a few more people to be
interested in the code, I think now is the best time to fix the
indenting. It's a large patch, but there should be no functional
changes.
I removed one unused function and renamed internal-only function to be
prefixed with '_'.
Change-Id: Ibb9b26f0cc8719be4fec121c1b9aa279938aebd7

It seems to be a duplicate and I don't think we need both.
Change-Id: Iaf7b6fe2b633fd6dd2d228960e48de238c8b1c33

This was renamed from drivers.cephfs to
drivers.cephfs.cephfs_native at the
request of the Manila maintainers.
Change-Id: I7ac2299ad323229418761bb06ee7891a48d87877

Currently glance v2 doesn't allow to specify custom
locations for image by default, it returns 403.
To enable this 'show_multiple_locations' param must
be set to True.
When Nova is doing snapshots with Ceph it provides
location to activate image, which fails when we use
Glance v2.
To fix it we change the default param to True.
'show_image_direct_url' will be overridden.
Related to blueprint use-glance-v2-api
Change-Id: Id0f1c398b8b48f2ffc2488b29bc7cbd279069337

manila's cephfs-native CI tests cephfs driver, which does not support
glusterfs protocol. The glusterfs protocol is enabled by a manila's
tempest config option by default. Modify the config option to disable
glusterfs protocol.
Change-Id: Ia7a226d6a8b0ba5aeedcdf34e8a6f9a8dbbfda40
Closes-Bug: #1541377 

The manila API tempest tests for consistency groups (CGs) also test
the 'create_consistency_group_from_snapshot' API that is not
implemented by the cephfs native driver. So disable it by default in
the CI hook script.
Change-Id: Ibb6b15a98970262fab4bd98ad2bd4acd3990747f

Use ceph.conf to set the default pool replica size, thus
avoiding the need to set replica size for each pool
separately.
Change-Id: Ib00c1666895af2332a3689ee59b9929af81a17e9

Set option to enable tempests tests related to 'cephx' access type
for gate-manila-tempest-dsvm-cephfs-native CI job.
Change-Id: If162271d81cd0d2efafd2d33659946eda30134ba

Change-Id: Ibdbf1feaf6a3452705b29a1865e319afba4d562e

When Manila is enabled with cephfs native driver, the plugin fails to
enable protocol 'cephfs' in Manila's configuration file. Correct
this failure.
Change-Id: I145c5628f781a807231bcf1a77d5e3010a7185ca
Closes-Bug: #1536625 

Change-Id: Ib62022f8d50901378b853dfd20d8d8341b3db10d

This patch enables the plugin to setup ceph radosgw and configures
it as a swift-compatible endpoint with keystone integration.
ENABLE_CEPH_RGW is the new variable introduced, which is False
by default, and can be set to True in localrc if radosgw needs
to be setup.
Fixed couple of other (related) things ...
1) Created rgw specific functions
2) Checking for radosgw & swift co-existence, erroring out early enough
NOTE: Tested on ubuntu trusty only
TODOs ...
1) Not tested on Fedora (F22), radosgw startup has some issues there.
Co-Authored-By: Deepak C Shetty <deepakcs@redhat.com>
Change-Id: I21db4168eb69d107599f6b6ab1668b02b764b2c6

The cleanup_ceph_embedded function nukes the ceph
conf directories, so when subsequently trying
to stop services they aren't stopped. Next stack.sh
run will fail because daemons are still running.
Fix by stopping daemons before running cleanup.
Change-Id: I51cd58eae83171a47aa898b65240b72ea0217af9
Closes-Bug: #1534600 

When Manila is to be configured with CephFS backend, explicitly
install `ceph-mds` package. The newer versions of Ceph packages,
do not have the dependency between `ceph` and `ceph-mds` packages.
Change-Id: Idfd4cf7fc017ad6ced2d325356ff97cc68889dc0
Closes-Bug: #1533314 

Let the plugin setup and configure CephFS as the storage backend for
Manila. This is not done by default. Refer the README to do so.
Add 'pre_test_hook' and 'post_test_hook' scripts that would be needed
to configure Manila-Ceph CI jobs.
The following changes are also made:
* Install a development version of Ceph that is compatible with Manila's
 CephFSNative driver.
* The development version is a version greater than Infernalis. And
 from Infernalis onwards, the ceph daemons are run as user "ceph".
 So allow the daemons to run as user "ceph" for such versions.
* Enhance get_ceph_version to check Ceph version even when the
 ceph mon daemon is not up, by checking the CLI version.
Co-Authored-By: John Spray <john.spray@redhat.com>
Change-Id: I74314bfcc6b52d524bb84f2232a988f275b9afbf

Tox 2.x does not parse the current tox file as is, the comments cause
"ValueError: No closing quotation". Remove the comments to make the file
work with recent tox.
This is caused by a backwards incompatible change in tox:
https://bitbucket.org/hpk42/tox/issues/181
Closes-Bug: #1527562
Change-Id: I2a12093dcb3207be1a888445ec59e1b7da849852

Add configurables so that one need not have Ceph be the storage
backend for Cinder, Cinder-backup, Glance and Nova.
Let Ceph be the default storage backend for Cinder, Cinder backup,
Glance, and Nova. Allow the default setting to be overriden when the
global config, ENABLE_CEPH_{CINDER,C_BAK,GLANCE,NOVA} is set to
False in the local.conf/localrc file.
Closes Bug: #1521278
Change-Id: I3afe7c57d5d56f15fe533ad5b1dbe36ccd369e6b

Now that DEVSTACK_GATE_CEPH is removed from the plugin job's
template definition [1], we no longer need this hack
[1]: https://review.openstack.org/251197
Change-Id: I92810ecc53af37b5c04db318bedf0fef571821e5

Update TODOs in README.md
Change-Id: I0e5ea5b610eafa04f43957e5e47e3a6eff87a273

Removes the broken fedora 20 url and ensures that ceph and
radosgw packages are installed from the distro repos. See
the associated LP bug 1521073 for more details.
Change-Id: Id11c51bd5451f5c7886696b519ad43f012e83d74
Closes-bug: 1521073

Fix README, as it was not rendering properly on GitHub
Change-Id: I9bda6c74ebb2ac35911231c53906b3e04bb860cc

Adding README.md for this repo
Change-Id: I2d03f0a92db5bb34fccf178239df76a0ff5d416b

Co-Authored-By: Deepak C Shetty <deepakcs@redhat.com>
Change-Id: I124b54b5068fbcd46e278281da12ea27f0034d1f