openstack/manila-tempest-plugin - manila-tempest-plugin - OpenDev: Free Software Needs Free Tools

Recently [1], we turned off a bunch of tests since we
were building out a way to run nfs-ganesha
standalone with a cephadm based ceph cluster [2].
Running API tests will increase the coverage. We
can retain the "backend" filter in our test regex
since we still don't want to run any API only tests
in this job.
[1] I5b7fd5b2b557203189c25fa2a988d790e7fda3eb
[2] If983bb5d5a5fc0c16c1cead84b5fa30ea961d21b
Change-Id: I2198eee3892b2bb0eb835ec66e21b708152b33a9
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

As per Ia1671de5c770d1cf5a3cd58e05fe5204f5bbc3c6,
fixed some ingress mode issues by identifying a
virtual ip as "CEPH_INGRESS_IP".
Bumping that variable to the parent job as well.
Change-Id: I8e04b866e2ebce291dfeb21dadda78ac2ecb8983

Reduce test concurrency to 1, and limit the tests to only those
tagged with 'backend' to hopefully reduce the resouce consumption
causing timeouts and flakiness in cephfs-nfs standalone jobs.
Change-Id: I777e8137c5a2b8f2a03667faf413ae65b1170d48

Neutron dynamic routing can be temporarily disabled, as it is not
being used until a known issue gets fixed [1].
[1] https://bugs.launchpad.net/manila/+bug/1940324
Related-Bug: #1940324
Change-Id: Iceeb41ff8c34fe86f2825089140801f983a96a85
Signed-off-by: Carlos Eduardo <ces.eduardo98@gmail.com>

cephfs-nfs job running on Reef, adding ingress ip
Change-Id: I5b7fd5b2b557203189c25fa2a988d790e7fda3eb

I28e6159944746abe2d320369249b87f1c4b9e24e introduced
OSCaaS and enabling it offers a good amount of speedup
in our test jobs. So lets enable it in the base jobs.
Change-Id: I4a1c8d75faa8151fb94cd227e24967a06d02ff4b
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

As 2024.1 is released, we should add its job on master
gate to keep branchless tempest plugins compatible to
stable branch.
At the same time, we're dropping the zed job
since the zed branch of openstack/manila has now
transitioned to "unmaintained". manila-tempest-plugin
may no longer support running against the zed branch.
Please use the "zed-last" tag instead [1]
Ref: Tempest plugins guide for stable branch testing:
- https://docs.openstack.org/tempest/latest/stable_branch_testing_policy.html
[1] https://review.opendev.org/c/openstack/releases/+/918346
Change-Id: I0625576d14d9e084aa65beed5d0c988dcebd85dc
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

This is an effort to remove package based deployment
of Ceph from devstack-plugin-ceph in favor of ceph
orchestrator (cephadm) deployed ceph.
Change-Id: I0c35ef0eb4293aae37cd597d78d7aed0100e8ec7
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

Share backup tests i.e. create/delete/get share_backups.
Change-Id: I366221702b3aa2e78bff82e6b2a543b6f6784f77

Xena and Yoga are past the stable maintenance
cycles. We're soon going to tag these release
branches as "unmaintained" branches [1]. We don't
expect the trunk version of manila-tempest-plugin
to work against these older branches any more.
Instead, we have tagged a "xena-last" and will
soon [2] tag a "yoga-last" branch that will be more
appropriate.
Going forward we must be testing the "maintained"
stable branches, which are zed, antelope, bobcat
as of this commit.
[1] https://docs.openstack.org/project-team-guide/stable-branches.html#unmaintained
[2] https://review.opendev.org/c/openstack/releases/+/908381
Change-Id: Ib0e984a4acdc2f82f5e3a75e34c7ec922674b0a5
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

Pop off ceph release pin in cephfs-native-cephadm job
Change-Id: Idf15ccf996eb6cdc05a960bdc6411f6fdb9e5fea

The scenario test
`test_write_data_to_share_created_from_snapshot`
was being skipped because of a bug in ceph's MDS
service. It has now been addressed [1]
[1] https://tracker.ceph.com/issues/57210
Closes-Bug: #1989273
Change-Id: I967479458038b67c7892a71ac881491e1f8c87f7
Depends-On: Iea541d2edefc871bcac2d965997c88462fcbe521
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

Use if=/dev/zero: Instead of reading from /dev/urandom,
which is slow for generating random data, use /dev/zero
to write zeros to the disk. Zeros are generated much faster.
Omit conv=fsync: The conv=fsync option forces dd to
synchronize the data after each write, which can be slow for
large writes.
Change-Id: Id845c60c39072a03171fa5906461eb254c2736a7

We had disabled scenario tests on the IPv6
test job manila-tempest-plugin-lvm [1] to prevent
gate failures at feature freeze. We can partially
re-enable these tests; while still working to
get IPv6 testing as well soon.
[1] https://review.opendev.org/c/openstack/manila-tempest-plugin/+/893651/
Related-bug: #1998489
Change-Id: Ie6599bc3533a002fa118ebe8b9243d677bdb410e
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

A multinode job creates a controller node
that runs manila with other OpenStack components;
and an additional compute node to spread the
compute VM workload around.
`manila-tempest-plugin-multinode-cephfs-nfs-cephadm`
replaces the existing `manila-tempest-plugin-cephfs-nfs`
in the check queue for this repository. These jobs
are different. `manila-tempest-plugin-cephfs-nfs`
uses devstack-plugin-ceph to install ceph and nfs-ganesha
as standalone components on the devstack host, whereas
`manila-tempest-plugin-multinode-cephfs-nfs-cephadm` uses
cephadm to deploy ceph, and it includes the deployment
of an nfs service. Manila's ceph driver communicates with
cephadm deployed nfs via ceph-mgr APIs, and with standalone
nfs-ganesha service via DBUS APIs. Ideally we'd run
both jobs all the time, however, since these jobs will
run individually against the manila repository, we'll just
pick the newer one here.
Depends-On: I84249ae268dfe00a112c67e5170b679acb318a25
Change-Id: I5162815b66d3f3e8cf8c1e246b61b0ea06c1a270
Co-Authored-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

Zuul has deprecated Perl-style syntax; and we're
getting warnings posted against the manifests:
 The RE2 syntax error is: invalid perl operator: (?!
Change-Id: I498573532fd701b47684a54bda9b54b86836afdc
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

A couple of issues happening at the moment:
- Nova recently merged a change that bumps the libvirt version in
the Bobcat release [1]. The new libvirt version is not available
in Ubuntu Focal. In order to get the new libvirt version, our jobs
that were using Ubuntu Focal, are now forced to use Jammy, so we
bump it to the new version in this patch.
- The support for Ubuntu Focal has also been dropped in this
release [2]. Making it one more reason to get the new jammy
images.
While upgrading the image, some other issues surfaced:
- Some CI jobs are doing IPv6 testing, which use quagga. Quagga
does not have a candidate for installation in Jammy and must be
replaced. This issue is already documented [3].
- The inability to set up IPv6, combined with the scenario tests
being enabled in the LVM and CEPH drivers brought up another known
issue: when doing IPv6 setup, we don't set up only IPv6 networking
and we also end up setting crucial information for the scenario
tests to run in the function that does IPv6 setup. So in case the
job is only using IPv4 in scenario testing, we currently don't set
some key configuration to let the shares be created/mounted [4].
- Due to the reasons above, the scenario tests will never pass, as
we can't use Focal images, and when using Jammy, we can't do
scenario testing.
This is blocking the Manila gate and all other changes in the CI.
As we are forced to use the Jammy image and the scenario tests
won't pass, we disable them in this change, do some refactoring
in Manila [5], and re-enable them again for IPv4, until we have
figured out the quagga replacement.
[1] https://review.opendev.org/c/openstack/nova/+/887255
[2] https://lists.openstack.org/pipermail/openstack-discuss/2023-September/034968.html
[3] https://bugs.launchpad.net/manila/+bug/1998489
[4] https://github.com/openstack/manila/blob/master/devstack/plugin.sh#L1104-L1116
[5] https://review.opendev.org/c/openstack/manila/+/893804
Change-Id: Ia45a8e7f7c9cf9959bcc9b9a7243cccbbd1f4916

We have a problem with glance's RBAC changes
in the generic driver. Disable the enforcement temporarily
while we explore a fix
Related-Bug: #2031191
Change-Id: Ia9c9eb4542c576e3e4d94334e62649abf3cb6faf
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

Package based installation of ceph via devstack-plugin-ceph
is deprecated. Recently, we broke support for Ubuntu 20.04 LTS
(Focal Fossa) [1] because we're trying to advance all OpenStack
CI jobs to Ubuntu 22.04/Jammy Jellyfish or Centos Stream 9. The
community's plan is to retire package based installation of ceph
on devstack during the 2023.2/Bobcat release cycle, and move all
efforts onto "cephadm" [2].
With CephFS/NFS, cephadm deployment works, but seems to require
more CI resources than a single test node can afford. So, we're
working in parallel to create a multi-node job [3][4] to ensure more
stable testing without foregoing coverage. As a temporary workaround,
we can run the ceph-nfs job (which doesn't use cephadm) on
centos-9 stream.
Unfortunately, ipv6 testing would have to be turned off because of
LP bug #1998489. We'll remedy this separately.
[1] https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/865315
[2] https://docs.ceph.com/en/quincy/cephadm/index.html
[3] https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/876747
[4] https://review.opendev.org/c/openstack/manila/+/869235
Change-Id: I6abf7a8eb275c5b29cfe29b3a0f54de76c5bb1e8
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

We are trying to drop testing the trunk branch (2023.2)
on Focal Fossa.
Change-Id: I66ac85b735566f8f7dd75234eb8e0be7773fee3a
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

Change-Id: If7bca173830d8d0912873d073d64c6251714817f
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

Add a new job for testing manila's Native CephFS driver
with a cephadm deployed Ceph cluster. This job inherits
from the existing "base" job and overrides the
ceph deployment strategy to use cephadm.
With this commit, we will use this new cephadm based job
instead of the package based deployment to test this
repository. As with the existing job, this new job is
non-voting as well. If the job's stable for a period of
time, we can decide to promote it to being "voting".
Once all CI consumers adopt the new cephadm job, we'll
rewrite the "base" job definition to only care about
cephadm based deployment.
Change-Id: I6e4db6b89f89d2374580827e2334635dc5ead5b9
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

We're running test_create_shrink_and_write
and test_create_extend_and_write tests twice,
changing only the client VM's IP address family.
Not much value in running all of these since we're
exhausting the disk space with these tests. We can
choose to re-enable these tests when the situation
changes with either the footprint of the tests
or the form factor of the devstack node.
Partial-Bug: #2009083
Change-Id: I868b2e40934e5d10eb0c5cc6fa867e6adc2cc9fc
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

As stable/wallaby is in EM phase, and currently broken
with the Tempest master. We are pinning tempest there (depends-on).
We are using the tempest 29.0.0 on stable/wallaby (depends-on) and
compatible tag for tempest 29.0.0 is manila-tempest-plugin 1.8.0
(for ddt version compatible with stable/wallaby constraints and tempest
version)
- https://github.com/openstack/manila-tempest-plugin/blob/1.8.0/requirements.txt
- https://github.com/openstack/requirements/blob/stable/wallaby/upper-constraints.txt#L80
Otherwise it fail for ddt incompatible version
- https://zuul.opendev.org/t/openstack/build/4dad3a22fe5d485a96722cb6d68da63d
Preparing the new set of jobs with manila-tempest-plugin pin to 1.8.0
which will be used to test the stable/wallaby.
Needed-By: https://review.opendev.org/c/openstack/manila/+/871864
Depends-On: https://review.opendev.org/c/openstack/devstack/+/871782
Change-Id: Ibf8c885529098417e8ff9b417bf172218f8d684a

As per the community wide goal to migrate the CI/CD from
Ubuntu Focal to Ubuntu jammy, devstack change was merged, image got
migrated on Nov 18. But Manila jobs are failing on Ubuntu Jammy.
To unblock the Manila CI we need to pin the jobs to ubuntu focal,
until all the jobs are fixed.
Change-Id: Ia235cf28382947a392151f2914ce2dda964b46a6

The tests validate policy share actions for admin,member
and reader users in a project scope.
This patch adds tests for the following resources/features:
- share
- share snapshot
- share type
- share network
- scheduler stats
Below settings were used for ease of code development
and testing:
1. DHHS = False
2. Dummy driver as it supports all features
3. Manila-tempest-plugin as a base repo to test the changes
Depends-On: https://review.opendev.org/c/openstack/manila/+/858761
Change-Id: I53bb9851445038f81032485817389ec31e2b7341

Change-Id: I44f0bf50c3d7042565dca54949965b2c4df1fb41
Closes-Bug: #1970661
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

We run IPv6 data-path tests with LVM and
Ceph-NFS drivers by setting up bgp to
advertise tenant network routes to the host.
Bug #1959666 prevents the use of OVN with
neutron-dynamic-routing. The underlying neutron
ML2 mechanism driver doesn't matter to
manila; so lets use OVS in the meanwhile with
these test jobs.
Closes-Bug: #1940324
Related-Bug: #1959666
Change-Id: I5b37f31e302ce75650f817a937d8f17ee58887b2
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

We'll need to pin manila-tempest-plugin in stable/victoria
when testing manila [1]; since the configuration is a special
snowflake for this branch alone, lets not carry it in this
branchless repository any more.
Depends-On: I23827477d3c25eb2b10bea6d62885d50a28ec0a0
[1] https://review.opendev.org/c/openstack/manila/+/850415
Change-Id: I6cfc413b3588e5d75a6ac6c9ebe6835541d4c4bc
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

Some recent changes introduced a branches variable into our yoga,
xena and wallaby jobs. Manila-tempest-plugin is branchless, so such
jobs should not have the branches variable assigned to them.
Change-Id: I8647d268ccd642def4a003f63ea26d804016c20f

Adds a new check job to test FIPS compatibility with Manila
upstream CI. We currently only have means to enable FIPS on CentOS
systems. Ubuntu support will come in a later effort.
Starting from CentOS 8, there is no candidate for a quagga package
installation. Our IPv6 setup currently depends on quagga. If we
keep IPv6 enabled, the job won't even start. For now, the job will
only set up IPv4 and run the tests for this specific IP version.
Co-authored-by: Ashley Rodriguez <ashrod98@gmail.com>
Co-authored-by: Vida Haririan <vhariria@redhat.com>
Change-Id: Ica6445c582ceae4c5b9bbce48702c74710e6056f

This changes addresses few comments left in the change that
refactored some of our CI jobs [1], regarding a more suggestive
anchor name and a comment to give more context for code reviewers
in the future.
[1] https: //review.opendev.org/c/openstack/manila-tempest-plugin/+/847366
Change-Id: I4abf0d60e6662c6f409bfa6506ab51ea717dc8cf

In order to switch our manila service image to Ubuntu Jammy, we
needed to get in a couple of changes that Modified the flavor for
the manila service image. After all changes got merged, we switched
to the Ubuntu Jammy image [1].
However, now the manila service image master on Tarballs is using
a Ubuntu Jammy image, which requires more resources. It would not
be ideal to backport the changes that modify the flavor to the
stable branches because it is not an actual requirement for them,
and that also forced us to reduce concurrency of some jobs on the
master branch.
Using the Ubuntu Jammy image broke our stable branches CI, since
with less resources we are not able to successfuly spawn a nova
instance using the Ubuntu Jammy image.
In the stable branches, the jobs that spawn VMs will need to use
Ubuntu focal, so we have a new couple of jobs that will
selectively run against the stable branches and they will use a
pinned version of Ubuntu Focal from Tarballs.
These jobs were grouped into project templates to help us keep our
manifests readable.
Change-Id: I14afe2572c5bc8064417ba7ba3be26751022901d

Due to some test constraints, we decided to migrate the
manila-service-image to Jammy Jellyfish [1]. In order to have the
jobs passing, we needed to grant more resources to the service VMs,
but it would not be ideal to backport those changes to the stable
branches.
We decided we would pin the manila-service-image to a ubuntu Focal
image in the stable branch, and with that we would not need to
backport the other changes we merged in order to switch the image.
[1] https://review.opendev.org/c/openstack/manila-image-elements/+/841264
Change-Id: I74f81e92297ad58ebd4266511bb62b5fe24f7cb5

We have been performing some tests using the Ubuntu Jammy image.
While doing such tests, we realized that the newest image would
need more RAM to be spawned successfuly, and that brings the need
to bump the memory for such VMs. While testing with the memory
bump, we realized that the generic driver would face issues with
such change.
We have seen a couple of tests failing due to the share manager
service loosing connection to the database, and this is usually
caused when we have a lack or high consumiption of resources in
a VM. Reducing the concurrency itself should work, and then
we would have the jobs passing. However, in reducing the
concurrency, this job will take longer to run, considering it
will only use one of the available threads, and it can lead to
this job's possibility to hit the 3 hours pre-scheduled limit.
To avoid that, we decided to:
Have this change to reduce the concurrency of the manila generic
driver job to 1, so we can avoid these failures, as well as
split the generic driver job into two jobs:
- manila-tempest-plugin-generic
- manila-tempest-plugin-generic-scenario
The generic one will take care of the API testing as we currently
know and have been doing. The scenario one will run all of the
scenario tests.
Change-Id: Id2d35ce592c73f0c731deb17a5d574f502fe64cb

stable/yoga is the newest stable branch, and needs
a job testing this branch. stable/victoria is in EM
state and we don't support using manila-tempest-plugin's
master branch against EM branches. Tempest is pinned
as well for this EM branch.
Change-Id: I949f6fc960c93084efec7ab36cd9f98b636513e5
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

Defining queues at project.pipeline level was deprecated in release
4.1.0 of Zuul [1] and support for it will (probably) be dropped in
release 7.0.0. [2][3]
[1] https://zuul-ci.org/docs/zuul/latest/releasenotes.html#relnotes-4-1-0-deprecation-notes
[2] https://lists.openstack.org/pipermail/openstack-discuss/2022-May/028603.html
[3] https://lists.zuul-ci.org/pipermail/zuul-discuss/2022-May/001801.html
Signed-off-by: Eduardo Santos <eduardo.experimental@gmail.com>
Change-Id: I63d331e575899fa962daf022eaf6cace96a703c7

This change adds negative and positive functional tests
for the new feature add multiple subnets per az. Older
functional tests impacted by the feature are fixed. Finally,
the new flags `run_share_server_multiple_subnets_tests` and
`run_network_allocation_update_tests` are added and configured
in the community jobs for Dummy DHSS=True and Container Driver.
Partially-Implements: blueprint multiple-share-network-subnets
Change-Id: I99547e1873646fb1494a454f67b14c7293342beb

stable/ussuri has now been transitioned to EM (Extended Maintenance) [1].
So we can stop testing manila-tempest-plugin changes against that branch,
and introduce a stable/xena job.
Dropping the stable/ussuri job does not mean that manila-tempest-plugin
does not support running against that branch. Extended maintenance
branches (stable/ussuri, train and stein at the moment) still get
per-commit testing when changes are posted to the manila repo against
those branches.
Change-Id: I1e019ec913f8933d50afae6739352686725f4780
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>

We're experiencing failures to SSH into test
VMs over IPv6 IP addresses from private tenant
networks in the LVM driver job when testing
main branch code. It's possible this is a regression
in nova, neutron, neutron dynamic routing or bgp. We'll
drop this test exclusion when the bug has been
identified and fixed.
Change-Id: Ib39cb2bda862758f575262647e068dd9deaf1459
Partial-Bug: #1940324
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>