openstack/openstack-ansible-os_tempest - openstack-ansible-os_tempest - OpenDev: Free Software Needs Free Tools

This patch fixes the warning:
[DEPRECATION WARNING]: The 'os_project_facts' module has been renamed to
'os_project_info', and the renamed one no longer returns ansible_facts.
Change-Id: I7fe4bda991b968530453c620e1ea56a187a26924

This patch add support to tempest run --worker-file option.
This add the possibility to manually schedule how tempest tests will be
executed in each worker
Change-Id: I9ea47452c430c6bf5ec0ac6332cecd5d3abc7792

Previously the tempest plugins were installed using pip independant
of the python_venv_build role, in the past OSA could support a mixed install
of a distro tempest package and source code plugins. The mixed mode has been
removed but the tempest & tempest plugins installation tasks remain seperate.
Tempest is installed using python_venv_build, and the plugins are installed
directly using the ansible pip module. Unfortunatly, these two approaches do
not handle openstack upper-constraints in the same way, particularly for a
package that is installed from source code.
The python_venv_build role will correctly install a version of tempest
corresponding to the specified git SHA even when u-c specifies an earlier version
of the package. Unfortunately, when installing the tempest plugins with
the pip module, the previously installed tempest packages gets downgraded to
match the version specified in u-c as the pip module is unaware of the combined
source + u-c constraints generated by python_venv_build.
Tempest is an unusual case, because it is perhaps the only "top level" openstack
projects which has its own constraint appear in u-c.
This patch also corrects the use use of the venv_wheel_build_enable variable.
As a role default the value of this variable is changed in the whole scope of
the role becasue it is overridden during the stackviz install. The approach
used should be reviewed and refined in a future patch.
This patch also marks bionic/distro jobs as non voting as there has not been
a release of tempest which passes keystone test_list_user_groups test.
Change-Id: I5e8c8540b30865aaccafe59c73fae6fdd6da80a8

Added tempest_tempestconf_profile_extras and
tempest_tempestconf_profile_overrides variables
tempest_tempestconf_profile_extras is designed to extend basic
tempest_tempestconf_profile dictionary when required. New var has
presedence in case of having same keys defined. It can be used
to define image or other extra params without need of overriding
defult things
tempest_tempestconf_profile_overrides var is a replacement for
tempest_tempest_conf_overrides which was used in inapropriate way
as tempestconf profile overrides have pretty different format from what
is used by config_template.
[1] https://docs.openstack.org/python-tempestconf/latest/user/profile.html
Related-Bug: #1868938
Change-Id: Ib1e78a6ce2085df82d2a22db6817e800432f2f05
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Co-authored-by: Dmitriy Rabotyagov (noonedeadpunk) <noonedeadpunk@ya.ru>

The "Move over workspace when config is updated" task was running if
copy_tempest_config changed, but copy_tempest_config is created in
"Copy tempest config" only if "not tempest_use_tempestconf" conditional
is met. To avoid an error like "'copy_tempest_config' is undefined", we
need to add the same conditional to the first mentioned task.
Change-Id: I0d603dfa7319d09201f5f1d64d10cc71c56c2c21

For ubuntu distro jobs there are no tempest plugins packages so drop the
'mixed setup' where tempest comes from a distro package and the plugins
come from pip, and then fail to install into the system python environment.
Change-Id: Ic146e011ec15889303061527e17cdc16ed914799

This patch unifies the master branch with the changes which have
been necessary to unbreak the stable branches in light of py2 support
being removed from tempest and its plugins.
Depends-On: https://review.opendev.org/704317
Change-Id: I1d447933af5b668c17576a0ed985683a28fa153a

Ubuntu does not package tempest plugins so even for a distro
install the plugins are installed using pip. We must use the correct
pip executable when installing the plugins for py3 environments.
Change-Id: Ib399176ad12d556790daf770f11e7e8e14c3fd98

The var tempest_mixed_setup was previously renamed tempest_plugin_install_source
but a couple of usages were missed. Correct this.
Change-Id: Ia7835491a967c1831f64213ba1b42779136a87da

Stackwiz install use venv_pip_install_args when defined.
Change-Id: Ib1cc8fbe721aff1840251e8858d9a4a7eaa1ab41
Close-bug: #1852276 

The blacklist defined for older release with in the job
does not have test attributes leading to break the role.
In order to have compatibility between older blacklist and
newer blacklist having test tag schema fixes the issue.
Change-Id: Ide422b7b60a0b3a9321fb4686f0a2ab922200b7e
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>

Fix the wrong word "visable" to "visible".
Change-Id: I13d77c0d24a5fbf9c7839b55918a7a886cc09256

As image should have been already cached in CI [1], we shouldn't
download it again but just verify the checksum.
[1] https://opendev.org/openstack/openstack-ansible/src/branch/master/tests/roles/bootstrap-host/templates/user_variables.aio.yml.j2#L226
Change-Id: Ib764c70e483513bec5bd97d4499e67beb4501826

In CI, when we run full tempest api and scenario tests, it takes
time and in most of the time at Execute tempest tasks hits
Timeout and no ara report got generated. So we have no idea
what tests ate the time.
Logging the tempest result in a file will give more info
on what happened with the tests.
Change-Id: Ibba761551e6bc14c127d9461de2a72e9f0fcb715
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>

Because of jinja2 templating delimiters inside when statement tempest
wasn't correctly installed from source.
Also condition is simplified to be more readable. install_method check
works as we would like to run venv installation only for source
installs, but not for the ubuntu mixed setup.
Depends-On: https://review.opendev.org/686041
Change-Id: I39332a35fb2edae65a4762fe24c5b1f8c8675a2b

Also allow a default var to be specified per distro, maintaining
the current functionality for tripleo without requiring an override.
Change-Id: I99e0082a748d62ed129a860c6f16c523dec8d50e

The vlan network provider, requires a physical network type, otherwise,
it will fail
Change-Id: I021015709c8254ff5936ad08d61e1e02ab58c44f

The var tempest_plugin_install_source is defined within the
os_tempest role so if it is used in the condition that includes
the python_venv_build role, it must be in scope for the python venv
build handler which runs in the outer containing playbook, and
this fails with a var not found error.
Change-Id: I7d1fd3b7d06d3a0c6d86750ed36cbe858190f87e

Since debian/ubuntu do not provide packages for tempest plugins, we are using
mixed setup. We install tempest as a distro package, while installing
tempest plugins from source, but without venv (for system python).
That guarantees that we have all required plugins and do tests.
To accomplish that variable tempest_plugin_install_source was
introduced.
Change-Id: I198190d601cf8a922c3afaa69a3c0e42ab075735

- scm: git is mandatory on requirements.yml when using git urls
- role_name is needed in meta
- centos was not listed as supported platform
- galaxy_tags should be used instead of categories
- comparison to empty string should be avoided
These where found while attempting to enable functional testing using
ansible-molecule.
Change-Id: I5f8087516628e20c2539552c653d4ea5361234ec
Needed-By: https://review.opendev.org/#/c/675615/ 

For non-integrated tests tempest_install_method is not defined,
so when python_venv_build tries to launch meta handler, it fails with
undefined variable. Once default is set, this will prevent problems
Depends-On: https://review.opendev.org/#/c/665396/
Change-Id: I3cdfe48cae28969dea381bfa325c4bb4769fb4d1

Switch to include_task to import_task and include_role to import
role in the role.
Change-Id: I6b2d2ceead9cc16afc5c7c141c400b8ee61a9da5
Signed-off-by: Chandan Kumar <chkumar@redhat.com>

We have tempest_pip_install_args to pass additional
args to python_venv_build role for tempest. Let's use it
instead of pip_install_options.
Change-Id: Ie0299fdd9147aaf8162c3d094d7d619e75652528
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>

Resolve an ansible 2.8 issue:
argument provider_segmentation_id is of type <type 'str'> and we
were unable to convert to int: <type 'str'> cannot be converted to an
int
Depends-On: https://review.opendev.org/#/c/660697/
Depends-On: https://review.opendev.org/#/c/660772/
Change-Id: Ic88bb7d6f36fe9ad9ff66bfb5119eee5a5f67cd2

Due to tempest plugin split, all the tempest plugin requirements
are kept in requirements.txt, not in test_requirements.txt, so
it does not make any sense to install it.
So removing the flag from the role.
Change-Id: I7600f1750538335c1348e7d042e6bf450910e12c
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>

Don't bother trying to ping the router address when the public
network type is 'local'. In calico, a local network type is used,
and the router is not set up on the host until an instance using
that network is created. Since this check occurs before the
tempest run (and before any instances are created), the gateway
is not expected to ping.
Change-Id: I47543778487b6a9a2972b29d270d199a4d1e9b61

Currently os_tempest is used by multiple deployement tools
and each of them have different methods of configuring
services and currently tempest_service_available_{service_name}
are setted based on group vars and group vars is not used at
all places which leads enable user to specifically define a
list of tempest_service_available_{service_name} which is too
hectic.
By introducing tempest_services:
 - aodh
 - neutron
will set those vars automatically and makes it easier for
handling tempest_service_availabe vars.
Depends-On: https://review.openstack.org/#/c/649653/
Change-Id: I8734f38206b73e6547a6ec545f07f625e844ed66
Signed-off-by: Chandan Kumar <chkumar@redhat.com>

Currently in Generate test-list file task, shell script used to
generate test-list gets cancatnates with the whitelist-file
options leading to saying whitelist file not found.
Removing - from jinja conditionals fixes the issue
Change-Id: I05f08cfbaca4d030d20079f22080340d6e1c7239
Signed-off-by: Chandan Kumar <chkumar@redhat.com>

The variable tempest_developer_mode and tempest_venv_download no
longer carry any meaning. This review changes os_tempest to do
the equivalent of what developer_mode was all the time, meaning
that it always builds the venv and never requires the repo
server, but it will use a repo server when available.
As part of this, we move the source build out of its own file
because it's now a single task to include the venv build role.
This is just to make it easier to follow the code.
Change-Id: I7ab81ab707a5576682abcbe9952d85e79a1f3856

Currently keystone_service_internaluri_insecure var is used for
doing disabling/enabling ssl, but it comes from os_keystone role
which is not used at all places. It adds a dependency on
os_tempest role to set the value. Moving to a new var allows
use to define one less var in os_tempest.
Change-Id: Id20d13ed3138e39a14a3f8c6a9c11a6c3d3566df

Currently Generate test-list file shell task is wrapped in a
single long script with jinja template which makes it hard to
read and it also leads to cause jinja issue, breaking it into
multiple lines improves it.
Change-Id: I17dd53dc583d75a3a19019ba4d7eac8be7077338

Stackviz installs from a tarball in order to make it happen quicker.
If we were to build it from git sources, then we'd have to build the
HTML page ourselves, which takes approximately ten minutes.
Due to the tarball URL being used as the package name, we're unable
to constrain it without making the venv build role a lot more complex.
To keep it simple, we just disable the wheel build process and install
it from the tarball instead.
Depends-On: https://review.openstack.org/644391
Change-Id: Ieb498c49377d77577c98ddb2d911066e44d8d9dd

https://git.openstack.org/cgit/openstack/
openstack-ansible-os_tempest/tree/tasks/tempest_run.yml#n24
Syntex is correct, but --serial flag was not getting exposed in
rocky/master os_tempest tempest_run playbook, It refactors
the shell script and nicer so that it works perfectly.
Change-Id: Ieca5103f803778a44bbb517954b56a9069babbdd

Currently os_tempest role checks tempest concurrency to greater
than 0. But this review https://review.openstack.org/#/c/641349/
sets default concurrency to zero so we donot need to check the
default concuurency to greater than 0.
https://github.com/mtreinish/stestr/pull/232 already checks the
concurrency value to int so we donot need to define check for int.
Change-Id: If53b59f3faede91e158cd544742ab37639800b57

Currently tempest related directories are created by root, which
makes the os_tempest needs to be runned as root which is
dangerous.
This changes allows to create the tempest related directories
as a normal user and use root user/owner where it is need in
the userspace.
Change-Id: I698495bdf915d108dc3978d898fb802e12ef1c96

Currently the ping test runs from the tempest_service_setup_host,
which may not have the correct package installed to execute it.
The ping package is installed on to the tempest host, and the
requirement to do the ping test is actually to verify connectivity
from the tempest host before running tempest itself.
This patch changes the ping test to run from the tempest host,
and because the test is primarily for gate testing, it is disabled
by default.
Change-Id: I3d1f9e17ef45b4427e8d3088bbcfa0c0df126c4a

When os_tempest is used locally or not in CI for
development and testing, it does not make any sense to generate
stackviz report each time. Adding tempest_run_stackviz makes sure
when it is enabled then it will run.
Change-Id: I4f9a6bcce7e1f4b7ebc873ee11f141d40f0d3e36

The private option on include role was never implemented and
will no longer be developed. This change removes the option
so ansible no longer raises a deprecation warning.
Change-Id: I8c769b24c41cbe4703336fdadac977c63a9cbef3
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

When updating variables that would result in a modified
tempest.conf, the /root/.tempest/etc/tempest.conf or
/etc/tempest.conf would get updated properly. A tempest
init happens after and does not copy those into an
existing workspace. This change allows backup workspaces
to be kept on each modification to keep from loosing any
existing workspaces.
Change-Id: Id2de9cbd0f06b73dead5298afb857df2700bab60
Closes-Bug: #1763336 

This reverts commit 1084cd9954.
As when we move tempest workspace to a different directory, the
directory is empty and tempest will fail to run any operation stating 
no .stestr.conf file found. In order to fix the issue, we need to do
tempest init.
Change-Id: Ia29398d3e82b880efe5e82febbc6dd6cf32b7450