openstack/devstack-plugin-ceph - devstack-plugin-ceph - OpenDev: Free Software Needs Free Tools

this change ensure that python3-logutils is removed so that
that it can be installed by pip later.
Before today in passing build, there were no python3-logutils pulled by neutron deps so
it was not installed or tried to uninstall so all good
- https://zuul.openstack.org/build/590c5996ca1b402486bfe1c7e1d08535/log/job-output.txt
But from today (10th Dec), python3-logutils pulled by neutron deps and failure started
- https://zuul.opendev.org/t/openstack/build/722c6caf8e454849b897a43bcf617dd2/log/job-output.txt#9419
The root cause of why this issue started happening today is not known. May be its
pecan===1.4.1 ? I8ee467bbb363f428a005f92554812bfdae95881a making it install but there
is no change for logutils as deps in pecan previous version 1.3.3 also[1]). Or it may
be cpeh/ubuntu packaging.
But it is clear that python3-logutils is coming from somewhere and causing gate blocker
in Nova gate, let's remove it and later we can find the root cause if anyone need this
package for their ceph job.
[1] https://github.com/pecan/pecan/blob/1.3.3/requirements.txt#L5
Closes-bug: #1954427
Change-Id: Icb63649b252fd6eb229adeae454b5ec3c6b79cad

Few URLs from some ganesha repositories were not corresponding
to actual valid URLs, where packages could be searched in
the repository.
These broken URLs have now been fixed.
Change-Id: If27b488cfec29731b74e7db774c4811b0e34c14e

As devstack-plugin-ceph is branched, we moved stable branch
jobs definition in their respective branch
- https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/786308
- https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/786309
- https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/786310
- https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/786311
But we did not cleanup the master version of job definition so that we
do not need to do it for future stable branches. When stable/xena is cut
from master the ./.zuul.yaml version of jobs in stable/xena has the stable
branches variant which is being picked up by zuul for stable branch gate
instead of their own version defined in their stable branch.
- https://zuul.opendev.org/t/openstack/build/bc088154e50440fe9a2cacf52426748a/log/zuul-info/inventory.yaml#241
And it started failing ceph job when ceph version was update for master and
same is being picked by stable branch testin so due to above branch matching
logic.
To avoid this in future when we will have stable/yoga or so we need to cleanup
this in master. Master repo will only keep the master version of jobs and no
stable branch jobs.
Other stable/rocky and older jobs will be moved to stable/stein so that we have
them there if anyone cut the branch.
Closes-Bug: #1945358
Change-Id: I7061f8d1491ff957452c9c777e40186a4e9c324e

Move tempest py3 jobs to latest released stable Ceph, which is
Pacific.
Change-Id: I6cd5424f023d2a895f8ed259d21944490bc1de5f

When we deploy devstack, we need to initialize the RBD pool.
Not doing so means that functionality like rbd trash purge
scheduling will not work correctly.
Ref: https://docs.ceph.com/en/latest/start/quick-rbd/
Change-Id: I5b0b3b83fb7ef805929fdcd106a5c8a988b05ec4

Remove the need of using FORCE_CEPH_INSTALL=yes
for CentOS8 Stream
Change-Id: I76c2c3805a8f606dca69a0f12aca226680fda573

Seems a trace from older code. It doesn't make
much sense to allow all Fedora versions that
matches f[0-9][0-9] and then check for specific
Fedora versions in the following lines.
Remove this check and just allow some specific
versions
Change-Id: Ie14a453f96689f574f1b388ab8f6e5467a59b7f7

Change-Id: I66b8412e61cec0b77ea63e4551ba3f082892bd38

This will change the version of Ceph from Octopus to Pacific and,
with it, the version of Ganesha from v3.3 to v3.5 which is the
version shipped into the Ceph Pacific container.
Change-Id: I1b31ef9dd13e1d56284f8d9f8be03e3fee0eb0a7

devstack-plugin-ceph is now branched;
these test hooks were maintained to
support legacy style zuul jobs; All
manila zuul jobs were ported to native
zuulv3 style jobs in the victoria cycle
and no longer rely on these files;
so we can remove these test hook files
from all three releases (victoria, wallaby,
xena/trunk).
Change-Id: I81e838bf13cb32503ee2b8becd06daea5ab01dfa
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

This changeset adds a config option (TEST_MASTER)
to toggle testing with the current Ceph master.
The builds are retrieved from Shaman, which is the
platform used by the Ceph community to submit their
latest builds.
The main goal of this change is to use this on our CI
and be able to catch bugs in OpenStack services that use
Ceph as a backend more quickly. Because of this, the
required non-voting job is also added.
Change-Id: Ibe26f89e7be3061cdb7acb441727efb2c396e29e

The patch bumps min version of tox to 3.18.0 python in order to
replace tox's whitelist_externals by allowlist_externals option:
https://github.com/tox-dev/tox/blob/master/docs/changelog.rst#v3180-2020年07月23日
Change-Id: I4bc8f70dd208e39c41d68bb82cf54ccaea9f6ebf

devstack-plugin-ceph is now branched and each branch
will maintain the respective branch specific code and
testing.
Change-Id: I9edf3ff59055e1ce6cd1630ea8e747ff5b498f02

The risk of breaking the gates by forgetting to run the jobs
against the old branches is too high
(it has happened with https://review.opendev.org/778960/ for example)
Related-Bug: #1921897
Change-Id: Ieab4e0ee482604802485cdf573c31e3c373a8553

In some jobs, we run tests that use "rbd map".
On older kernels, this will not work if newer
RBD features are used. (Specifically, fast-diff
on bionic.)
On older distros, enable fewer features by default.
Closes-Bug: #1921897
Change-Id: Iff1c5210b0279379eb9612b206bdf456dd6f132e

Change-Id: I59c8692ed7dab123c5bfce9e7e28646b4800846f

This change introduces a multinode ceph job based on
tempest-multinode-full-py3 to be used to provide live migration and
other multinode coverage initially in the openstack/nova project.
As this job uses ceph for both Nova ephemeral
([libvirt]/imagebackend=rbd) and Cinder volume storage the live block
migration tests are skipped.
Change-Id: I9ffdff44a3ad42ebdf26ab72e24dfe3b12b1ef8b

This change stops configure_ceph_nova from creating the vms pool when
CEPH_REMOTE=true as this suggests that it has already been created by
the controller that has CEPH_REMOTE=false set.
Change-Id: Iaad67025a23bf29e29011c6e78662692af2b564a

The typical Cinder deployment should have
more than just "layering" enabled.
Enable fast-diff, which requires exclusive-lock
and object-map.
Previously, there was code that tried to enable
exclusive-lock when CEPH_ISCSI is enabled, but this
code would not work reliably, since it was doing
addition instead of bitwise-or. Just remove it
since exclusive-lock will be enabled by default.
Change-Id: Idad98cfebf5e3b265316f2b6a3c201b472faa32c

Update the list of tested stable branches (excluding stein which is EM).
The nodesets for the stable branches are now pinned because
their testing environment are well defined and won't change.
Also, remove the -py3 suffix: those jobs are all -py3 based
(the suffix was mostly requires in the transitional time
between py2 and py3).
Change-Id: I48f8574960df51fc47f80fa17091fd13ec71aa99

Consume packages from download.ceph.com for ceph octopus
Our focal fossa based CI jobs are getting octopus from
ubuntu ppas instead of getting the packages from
download.ceph.com.
This happens because, even though we were defaulting to
nautilus, ubuntu got the later version available
(in this case, the packages in ubuntu ppas).
Also, remove CEPH_STABLE_RELEASE env var, unify to CEPH_RELEASE
Stable branches CI versioning is done in the jobs definition now
Change-Id: I1edd5a50079f325fa143a7e0d51b3aa3bb5ed45d

Having debug_ms enabled populates logs with too much
information. This makes it harder to debug in most cases.
Disable by default.
Change-Id: If180f9be42e95492ab457420e48a494fd2689e6e

As with tempest.api.network the identity tests have nothing to do with
ceph so lets blacklist them and hopefully reduce runtime.
Change-Id: Ibcda268021b236bc049e21b7ddd9e99a7c24dace

Add a glob pattern at the end of the file removals so that we
also capture items like
 nfs-ganesha-ubuntu-libntirpc-3_0-focal.list.save
which can otherwise cause issues next time we stack
(install_prereqs sees it and gets confused).
Change-Id: I6bb75a75972c47fa87f0d703dda242d2b9bb07b7

When working locally it is nice to be able to
run unstack.sh and not have too much extra stuff
to clean up by hand before running stack.sh again.
So clean up CEPH_DATA_DIR, contenta and mounts.
Also fix a spot where '/var/lib/ceph' (the default
value for CEPH_DATA_DIR) was hard-coded.
Change-Id: I43ff06ca38f42ea4305b2c862902bf6f612e32f0

To use the snapshots capability that the cephfs drivers have
we need to set the share type extra spec enable_snapshot support
to True
Change-Id: I45dddd49578eaca81276bbf44a44701c80b8a8c4

We are using NFS Ganesha 2.7 which is not longer supported
by the Ganesha community. Let's use v3.0 instead.
Closes-Bug: #1898952
Change-Id: I69543fd33c98817bba76c6712ae91264c532c12c

Disable caching done by ganesha when ganesha
is used as CephFS's gateway (manila's usecase)
This is the recommended Ganesha setting as libcephfs
already caches information. Doing so helps avoid
cache incoherency issues.
Change-Id: Iacd575ac61b39ab9dcf8814514b5e3267570a17f
Closes-Bug: #1898950 

We create a disk image during setup, which now gets persisted in
/etc/fstab across boots. Clean that up when we're instructed to
do a clean operation.
Depends-On: https://review.opendev.org/#/c/743007/
Change-Id: I54a6b637266c2f182eeece8942156b07581a7f0b

I'm not sure why this isn't failing in the gate, but locally I get
100% fail if I don't add the release key before the apt-add-
repository. This seems to be a change in focal as it worked fine
for me before upgrade.
This moves the key fetch/add before the repo.
Change-Id: I50d840986d630af033a7a3e47bb1e9c52e889791

This patch updates the override-defaults to support
setting the CINDER_DRIVER, CINDER_ENABLED_BACKENDS
and CINDER_ISCSI_HELPER env vars for devstack to
ceph_iscsi if and only if ENABLE_CEPH_ISCSI is True.
This is needed by devstack lib/cinder to automatically
configure and enable ceph_iscsi as a backend.
Depends-On: https://review.opendev.org/#/c/662829/
Change-Id: I3447554343edfb5ec84ea65a521168e6d2831dfa

Change-Id: Ic301602c7aada2b64d119b829b6919ee93a5870f

Remove a limitation from the README that's
not relevant any more and stop rendering
the search page in the PDF.
Change-Id: I4c3409c9a6384adf0c20ab08214b866063d4dea5
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

Given the issues introduced by I0eab8ad25112511b0c4dd4b7796b4cda821bca8b
it would be nice to have a simple RPM based job somewhere in the check
queue to avoid a repeat in the future.
Change-Id: I601b77f2da0574d38ccef7d05db4933f6fdae998

This patch enables building .pdf documentation using new 'pdf-docs'
tox environment. The newly created tox environment creates latex
source codes using sphinx-build and then builds the pdf
documentation using make.
Also, convert README.md to README.rst for more seamless integration
of the README file in the pdf and html documentation.
Story: 2006070
Task: 35457
Change-Id: I5c022478c9be0067c125efb4032c16cf0978330f