openstack/puppet-openstack-integration - puppet-openstack-integration - OpenDev: Free Software Needs Free Tools

Stackviz will be useful to monitor performance and debug our Tempest
jobs.
See https://github.com/openstack/stackviz
Change-Id: Ia909a8da9975b465f3d94de0cdaea56529bb325e

This reverts commit 38dd1cfcfa.
The review has been merged upstream.
Change-Id: I3071819b48b59c786ab7d7f9a208331dddccf42d

Change-Id: Iac7a5821b51c7603671181fcccb1e63428c6be53

* drop workaround to deploy Gabbi from pip, and use RDO package.
* Add gnocchi in the Puppet Tempest interface.
* Add gnocchi tests in the list of tests we want to run.
Change-Id: I51094b28c42f7c64d4357f49d0932e027cbd6310

Stop the script if Puppet fails to deploy repositories.
It can happens in case of repo downstimes or timeouts.
Change-Id: I01326be2bc4ab8380ce2a371a6def38011a0523f

Change-Id: I78324c8787b129e91f1ac32694085bde5b6043fe

checking selinuxenabled exit code before will ensure that the script does not
exit prematurely, since the bash -e flag is set
Change-Id: Id4f0831612f69f448e627453352bd88ce089da2f

Change-Id: Iefed86a09eaf4e571d38c801f79e9fc989bc57d6

PUPPET_VERSION is already used by puppetlabs/beaker and we don't want to
overlap with this variable. So let's rename it to PUPPET_MAJ_VERSION to
avoid strange behaviors when running Beaker jobs.
Change-Id: I74be2eec2be53f59af196f37ee348ff6856af8bc

* Pin Tempest on Ubuntu. Latest Mitaka release on Xenial broke Tempest
 tests.
* Bump RDO to latest repo so we can test Octavia & also unpin Tempest
 (Ceilometer Tempest tests were broken).
* Unpin Tempest for centos7 jobs.
Change-Id: Id550290bf9ebb2c6190072b3e8a4c501f535b2d5

See context: https://review.openstack.org/#/c/325520/
Change-Id: If00246874a66b73d79affc9cef56e71088b534f9

Latest version of RabbitMQ is trying to write in SSL certificate. Until
we figure where is the problem and how to fix it, let's ignore this AVC.
See https://bugzilla.redhat.com/show_bug.cgi?id=1341738
Change-Id: Id02038756b81c16fe6a64a54cb56f7bcee3588fc

https://review.openstack.org/322608 has been merged so Ironic tests
should pass again. Let's bump RDO repos to try latest OpenStack and
unpin tempest so we run it from master.
Since we still deploy Mitaka on Ubuntu, disable Ironic testing on Ubuntu
because it's impossible to run Tempest tests from master using Mitaka
codebase.
Change-Id: I15f52f0e21cb3dae84fa54d7067157ada726533a

See whole context:
https://review.openstack.org/322608
Until the patch is merged in Ironic, let's bump Tempest so our CI keeps
rolling.
Change-Id: I13baf13d5b77263a96699b11f38c9257aec67a67

Add support for Xenial deployments.
* Disable SSL on Ubuntu Xenial deployments.
* Pin puppetlabs-mysql to a recent commit (until they produce a new
 tag).
* Disable Ceph on Ubuntu Xenial deployments, neither Hammer or Jewel currently
 work out of the box with puppet-ceph on Xenial.
* Disable Horizon on Ubuntu Xenial deployments: not working yet.
* Do not deploy UCA on Xenial, since we currently deploy Mitaka.
* On Xenial, deploy Puppet using distro packaging, not puppetlabs.com.
* Cleanup /etc/default/puppet on Xenial / Puppet4 runs (workaround)
 until we figure what is installing this file on Xenial nodepool
 image.
All issues are documented here:
See https://etherpad.openstack.org/p/puppet-openstack-xenial
Change-Id: Ifc8c7e241add830c88b84c1f01770e5e15df1653

https://review.openstack.org/321100 has been merged, we should be able
to deploy Tempest on master.
This patch bumps RDO repo to have the latest commit in Ceilometer and
also unpin Tempest.
Change-Id: I430518c4336a5eb4cd0caedb1eb637783303e1d3

This patch squash 2 commits to bring our CI back:
1) Pin Tempest to latest working commit:
https://review.openstack.org/#/c/297511/ broke us, we need to pin
Tempest to keep rolling.
This will be reverted once we sort things out.
2) rabbitmq: install packages from distro repos
Stop installing RabbitMQ from rabbitmq.com repository.
rabbitmq.com is providing a terrible and frustrating experience of
packaging repository:
* rabbitmq.com is very often down and we have to recheck our jobs
 everytime it happens.
* packaging signature is currently broken, see [1].
From this patch, we'll deploy RabbitMQ using distro repositories.
[1] https://github.com/puppetlabs/puppetlabs-rabbitmq/pull/476
Change-Id: I1e79c04d7cd796493ae56423b70626e8841cf284

This reverts commit 0681b75c39.
Change-Id: I42b059cc774970c9eaa13e02c5f66a474dad3173
Signed-off-by: Paul Belanger <pabelanger@redhat.com>

1) scenario001: disable ceilometer tests on Ubuntu
Because we need this backport in UCA:
https://review.openstack.org/#/c/318503/
And we won't have it before... who knows?
Let's disable the tests for Ceilometer on Ubuntu until UCA provides a
packaging update.
Let's also disable Telemetry at all on Ubuntu, since Ceilometer can't be
tested until UCA does the backport.
2) add RDO Newton dependencies repository, so we have paramiko >= 2.0
and have a more stable gate.
3) Update RDO trunk URL to have a very recent URL.
4) Update nova endpoints to match with what is required upstream (2.1)
5) Un-pin Tempest, so we keep testing master (our desire in our master
gate but we pin it in stable branches).
Change-Id: I10dc1beeca3ec47fd4c3a866aa3e1d3bc9d0aa81

If found, create a symlink for tempest. This will stop the need for
downloading the image for every job.
Change-Id: I1986b8c6849f5a18b83245e927bfff513ce5f84f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>

Ceilometer tempest plugin is broken since:
e4796f8de4
Until we fix Ceilometer, we need to pin Tempest otherwise our gate is
broken.
Change-Id: I8eb0a3e2b27011c3d791c03fa02dcafb4fab081f

* Pin RDO to latest Newton.
* Pin UCA to stable repos for Mitaka. Canonical plans to provide Newton
 repos after OpenStack Summit in Austin.
* Install Gabbi manually, required to run Gnocchi tests.
Change-Id: Ic36fd606fe06202b0ca5b8eeaf5c5bdc2a5708fd

Add more debug when Puppet fails during first and second run.
It will help people debugging CI to figure why job failed.
Change-Id: I9ef32afaa594a65be6ed49f40bd8fba735fe2c70

In order to help be able to determine what part of the script a failure
occurs in, this change adds a print header function and updates the
shell scripts to print some header information for various actions.
Change-Id: I8e622c4d1c9c03aa3dbcb27712a346ce3284003f

Stop to run mistral tests for now, they fail too often.
We'll enable it as soon as we find what's wrong.
Change-Id: I041b6d35708c10df6bbddf672f56deb56ff86ab7

Do not merge, testing with v3 auth
* Deploy Mistral on scenario003
* Run Tempest tests for Mistral
* Bump repo to recent RDO repo that contains a fix in Mistral to make
 Tempest tests working.
Change-Id: Ia43427cc207ee5b7b95e9610f06a568992fb58cd

Run the tests in permissive mode but run sealert at the end to see if we
have some SElinux alerts that we need to fix.
We want to run catch_selinux_alerts function before exiting the script
(with failure or success) so we can detect alerts:
* right after first Puppet run
* after second Puppet run
* after Tempest run
If an AVC is detected, fails so we can report the bugs in
openstack-selinux and bring useful feedback.
Change-Id: Iaef45c88232505600cbb80f82b195aafcc615474

SElinux tools will be useful for us to provide SElinux alerts reports
during our CI jobs.
Change-Id: Icce81078d91bbed12b9dbba3c53416476d594a09

This is an attempt to reduce the console log size.
Change-Id: I24ac9ee5365405ca18670890035ac96e79dc9180

This is a workaround until https://review.openstack.org/#/c/304399/ will
be merged and a new image will be built and uploaded to OpenStack Infra
Clouds.
Until then, we'll carry this code so we can move forward with Puppet4
testing and deploy Facter correctly.
Change-Id: I823087aed543c3cf43d247f7adf5f32c5c0fc31e

epel-release is installed by OpenStack Infra when building images.
The problem is RDO + EPEL does not work find, there are some
dependencies issues.
This patch makes sure EPEL is not installed when we start our CI jobs.
The next steps will be to run puppet-openstack-integration with EPEL
disabled in ceph::repo, and patch ceph::repo to use centos ceph repos [1]
[1] https://bugs.launchpad.net/puppet-ceph/+bug/1562998
This patch is part of the efforts to stop using EPEL at all.
Change-Id: I506754e2c5021ee728675a8771f2e24b8576f7d5

puppet-zaqar is freshly new and we want to test it.
* configure mongodb replicaset (required for zaqar): 'openstack'.
* deploy zaqar on scenario002 with mongodb backend.
* configure tempest to test zaqar.
* run TestManageQueue tempest tests in tempest.
Note about TestManageQueue:
Zaqar does not maintain Tempest tests, and run their own functional
testing tools. api.messaging.test_queues tests are known to work, so
they will be the only tests run in our CI to validate Zaqar is working.
Note:
* SSL does not work out of the box, we need to add support for WSGI
 deployment with Apache, but Zaqa does not provide WSGI app, it's in
 their roadmap.
* IPv6 binding on default wsgi does not work, so using IPv4 now.
Change-Id: I1aa563f37aa7a231b2218f79b72c73c668d968f3

For debug purpose, it will be useful to have the list of tempest plugins
that we have.
Change-Id: Ib60dd99506f273f6759475ed49a57af78a6d0213

We want to run Tempest tests for a maximum of projects we're testing.
Tests are moving from Tempest tree to projects tree.
It takes care of installing tempest-lib which is required system wide to
run Tempest plugin tests.
To run tests from projects tree, we need to run tox with -eall-plugin
option.
This patch aims to change the way we run tox.
It also set manage_tests_packages to True so puppet-tempest will take
care of packaging dependencies required to run Tempest plugins tests in
RDO.
Depends-On: Iac9a57c2ba006bb2a660d2b54ee05bbe68125abf
Depends-On: I8f26968b5b817ffbb9f4dc4a0cb19ca0a1ef7d3d
Change-Id: I0942a7c9456d18528a2d84e143ae056973a7674f

A typo was introduced recently and this patch re-enable the testing of
Horizon.
Change-Id: I7520e862f9028baac23dbe92ccda471cd716ec77

This commit dynamically discovers the path to the puppet exucatable so
 we can leverage it when using sudo.
 This is to work around that the puppet application lives in a location
 outside sudo's "secure_path."
Change-Id: Ie54cdf90a8e21ea0de4df1eca31048db1ffb884b

This commit puts in place the ability to run
 puppet-openstack-integation against Pupept 4 by making release files
 and paths dynamic based off an environment variable, PUPPET_VERSION.
 Configurability is required because of the way Puppet 4 is installed,
 as an all-in-one package known as "PC1" or "puppet-agent" which
 includes Ruby, Facter 3, and other dependencies.
Change-Id: Id3af52f4938378a5a620765321cecd547e079660

* Deploy Self-Signed Certificates for both IPv6 & IPv4 deployments.
* Disable IPv6 for RabbitMQ now, for SSL reasons, will be enabled again
 later in a next iteration.
* Deploy Ironic API under WSGI instead of eventlet.
* Switch Glance API, Ironic API and Keystone to SSL.
* Configure Tempest with SSL endpoints when needed.
* Reduce the Ironic tests because of [1].
[1] https://bugs.launchpad.net/ironic/+bug/1554237
Note #1: puppet-swift, and puppet-cinder will require some work to support SSL, so it's not
implemented in this patch.
Note #2: we don't enable SSL for Neutron because of
https://bugs.launchpad.net/neutron/+bug/1514424
Change-Id: Ib2b5289b6f5e82f43cf60dee3152b2c2ddd5a014

avoid fatal error: `destination path 'XXX' already exists`,
and prevent package download failure.
Change-Id: I5ca84f43304397cb2017131ead6f321787ae09e1
Signed-off-by: guessi <guessi@gmail.com>