openstack/puppet-openstack-integration - puppet-openstack-integration - OpenDev: Free Software Needs Free Tools

Following the efforts to use more mirrors, this one is for using
buildlogs mirrors from OpenStack Infra, so we don't rely on centos.org
infra anymore from now.
Change-Id: I6c4db33f246aaa037b774b8af5ebf99d04022658

Instead of processing some variables, stop loading deprecated
/etc/nodepool/provider and use /etc/ci/mirror_info.sh to get some
variables for free.
Change-Id: Ic7c475eaa314ce2458ba2e9ca3f4683033bc2b64

1) Generate HTML page from Tempest results, using subunit2html
2) Compress and expose both subunit & HTML results into logdir
Change-Id: I111eb8417428ce2027d21ccdcdfd10dc30d509ac

* p-o-i ci deploys the cloud using rpm and installing packages
 from pip as sudo user may screw up the environment.
* we run tempest from source and testrepository is a tempest
 requirements. So we can use the same to generate subunit file.
* Cleaned up repeated tempest/testr binary by moving to a variable.
* Install testrepository on debian only.
Change-Id: I0387855dd39776d69c62b175a3eb39b84455b927

Improving scenario001 use-case where we use Telemetry services for
Auto-Scaling. It's a end-to-end scenario, so it will really test all the
stack.
- It adds Heat to scenario001
- It runs TestAutoscalingGabbi test
- It increases token expiration to 2400
Change-Id: Ibaa44cf53b8c8fb59b765ce4f9b219dd65f38a42

This submission will enable L2GW on scenario004
and one of its tempest test.
Change-Id: I87fb7ca3671cf93d6b47ac393e047e1810daf375
Signed-off-by: Ricardo Noriega <rnoriega@redhat.com>

Currently, tempest plugins are installed via rpms and
are used(as --system-site-packages is used during virtualenv creation)
in tempest virtualenv.
Tempest installation is not currently constrained, this patch
makes it constrainted to upper-constraints.txt from [0] to avoid
dependency errors.
[0] https://github.com/openstack/requirements
Change-Id: I85510dda99e600226f78f8cbc1361e099b68a7c9

Allow to deploy Puppet 5 (experimental) when setting
PUPPET_MAJ_VERSION=5 (currently done by experimental jobs).
It will pull packages from nightly builds, so we can use it and see if
things are working.
Change-Id: I3d0f2f4309c425435370d7f7d4d367c62396c8fc

Currently, p-o-i allways tries to install puppet package
from puppetlabs repo. In some cases, as RDO-CI we use p-o-i
to test puppet modules from RDO distro and it's recommended
to test it using puppet version provided in RDO repos which
uses different paths.
This patch provides a way to override this behavior by using
env variables as:
MANAGE_REPOS=false
PUPPET_PKG=puppet
PUPPETFILE_DIR=/etc/puppet/modules
Change-Id: I680504ab65d780deba53387e9b6604d3102de47b

Puppet 3 was EOL as of December 31, 2016. Since we don't use it in the
gate for Pike, let's drop the configuration bits in p-o-i for it. This
change leaves the PUPPET_MAJ_VERSION support on the off chance we need
it in the future but basically 4 is currently the only supported
version.
Change-Id: I0f04a3cb194ec2225010fb4e062703195b79baf1

While testing incoming release of eventlet 0.20.1 we've
discovered that it's increasing memory usage for processes
using eventlet. It has been reported upstream [1] but
aparently this is expected.
p-o-i scenario003 is pretty heavy in terms of memory. Before new
eventlet version it's already using ~700-800MB swap in my tests and
with new eventlet (not pushed to RDO Testing repo yet) it has failed
with out-of-memory problems from time to time.
With this patch, i'm increasing swap size from 2GB to 4GB to avoid
oom problems. Note that this may derive in performance issues if
we start having too much swap usage, but i haven't hit it in my
local tests.
[1] https://github.com/eventlet/eventlet/issues/412
Change-Id: If333ae62dbae3f597d2b14ae1dd4c174ecdf51fc

2 tests are timeouting too often and it makes our CI unstable.
Let's ignore the tests now.
Change-Id: I0140e774c0ee27926f03c01a972fb76721e71528

This submission will enable the BGPVPN API
on scenario004 and one of its tempest test.
Change-Id: I9d7853bedd822a9d182f1c5dcd7db85e7cd59c07
Signed-off-by: Ricardo Noriega <rnoriega@redhat.com>

In newer versions of the puppetlabs provided puppet-agent, some of the
tooling has been moved to /opt/puppetlabs/puppet/bin. We need to include
this so that things like gem contianue to work.
Change-Id: I4296ed611830af581d64777377c87a51230241dc

Mistral running on mod_wsgi should only have 1 thread per worker. This
sets threads to 1 and reenabled all Mistral smoke tests.
Change-Id: I2f423b42cbff35449e8f59dcbde113dc88038f78
Closes-Bug: #1671505 

Before create and add swap on system, check whether swapfile
is mounted. If so, try to swapoff the swapfile and remove it.
Change-Id: Ic3a9ffc34872a83a491412237369442f369c21d1

Currently, openstack-infra has create an proxy cache of
trunk.rdoproject.org. This is a short term solutions until we add RDO
to our AFS mirrors. Update repo.pp to start using it.
We also use the direct hash from delorean.repo, to avoid potential
issue where packages chages on remote node after we start downloading
things.
Change-Id: Id1819cf0dc49a89256a971ce9d8420622db901e8
Signed-off-by: Paul Belanger <pabelanger@redhat.com>

Until now, we were only supporting the Ceph mirrors for Ubuntu platform.
- Enforce http for OpenStack Infra AFS mirrors (https is not supported
 yet)
- Set CEPH_MIRROR_HOST to the right mirror depending of the platform
- Force https when possible
Change-Id: Ica91b23c0954ccee17ea13903048b6133758f9f3

choose a test without heat
checking a basic functionality of vitrage
sending an event to raise an alarm and see it in vitrage
Change-Id: I0bc2eb459e1ac4458833bac2d319a94b23b98b6a

We're now well past ocata-m3, so lets remove the exclusions we had for
issues with ocata ubuntu packages.
Change-Id: I7af84157a6c91c58981108747549c15ff8feb80a

We are hitting timeouts in IO bound processes as swift but we
don't have much information about what processes are causing
this IO peaks.
This patch adds iotop and iostat information to the logs in
addition to dstat output.
Change-Id: I4c50d4405a923eb2652fc39f6cea9889597ae44e

CI is currently flacky / broken.
1. ubuntu: exclude test_create_show_delete_firewall from tempest
f2088f3b7b
This patch broke us when running Tempest with Ubuntu Ocata.
 File "/usr/lib/python2.7/dist-packages/neutron_fwaas/tests/tempest_plugin/tests/api/test_fwaas_extensions.py", line 102, in _wait_until_ready
 if not test.call_until_true(_wait, CONF.network.build_timeout,
AttributeError: 'module' object has no attribute 'call_until_true'
2. Start Designate services after RabbitMQ resources are ready
This makes sure Designate services can start correctly at first Puppet
run and make second Puppet run idempotent.
3. Disable vitrage which is unstable now, until we figure out what's
wrong with configuration. It seems related to auth parameters.
Change-Id: I84921cbf4f4f838bff4f970113e709181801bed9

This reverts commit 9919ccfbb0.
Change-Id: Ieb444340b15b2d6866f82785d13c9143c7d050a6

ceilometer-api is being deprecated, so we want to stop running the tests
and move forward with Gnocchi tests, with auto-scaling (coming in future
patches).
Removing TelemetryNotificationAPITest that is not a valid use case
anymore when using Gnocchi.
Change-Id: I0e09dea793e86a6be40a2ccc4715a1dc9794713e

Tempest recently merged a change
I54a001cb562a8aac91537bf61e82a7e7d3498788 removing an option that
our designate packages still have. Lets pin until we promote.
Change-Id: Ifa1663f3b8f6cf51fb9911349885aa0ba6b9205a

Change-Id: Ibdb1e687ec348fa4147a9ca970b4f0f1c355ade9

Tempest-style tests have been migrated to tempest.lib, part of tempest.
Change-Id: I46e33abfaffd8d78240376b272a75f63f2c37e13

* TEMPEST_FROM_SOURCE is set to true to install tempest from source.
* Added capability to use tempest rpm
* change the cirros image directory to /tmp/openstack/image
 if the image gets firstly downloaded in tempest workspace,
 Tempest init will fail to create tempest workspace.
* Create image_dir if not exists.
Change-Id: I349783ace314a3575e93e6c853a7756f0f9b85ac

Since ocata-m2 has been published in proposed for UCA, let's see what
tests still fail.
Depends-On: I272c03f19d7182c4a731a1ef0fcbc2b9c6ca0b97
Change-Id: I4c8e102dea02e931abdc708f6ad2c508765a4c5b

When I run p-o-i on my laptop, logs go into / which is weird.
Default $WORKSPACE to /tmp, so when we run the scripts outside OpenStack
Infra environment, logs are visible in a safe place.
Change-Id: Ib8b0f9417098bcf703e31984804f2b72a8d93866

This reverts commit 4d6b918cef.
It activates Mistral testing again, module one test that fails all the
time:
https://bugs.launchpad.net/mistral/+bug/1654555
Change-Id: I763fd9b13ba94d1e76682a7a1c4264d7507c922c

Glance registry is not required for the v2 of the API and there are
plans to deprecate it in the glance community. Allow for it to be
disabled in tests.
Change-Id: I89f319f18f4cbda4244e96732a16df81a0f836b0

Tempest has been fixed upstream. We shouldn't have to pin it anymore.
Change-Id: I14ceda3ebad7797cbd278199ea3c3bc8151dbf66

We don't want to pin Tempest if we're running inside the Tempest gate.
This allows their patchsets to be properly tested inside the p-o-i jobs
that are running there.
If we're not running inside the Tempest gate, pin to whatever version
is in TEMPEST_VERSION.
Change-Id: If2221480fb13a69e64adc26fa6bc211bf563bfa8

Do not pin Tempest if the jobs are run in Tempest gate, so they can
actually see the errors we have without the pin.
Change-Id: I439dc6b8200a59f31c0227a124b4a2a6e8d443ba

openstack/tempest/master interface breaks Trove and more services.
There is an attempt to fix it here:
https://review.openstack.org/#/c/412511/
Let's pin Tempest until the fix works and is merged.
Change-Id: Iadf7c3164d2f16c5caa1b5d53089f35b22b7b289

This is a partial revert of the pin we implemented as part of
I2330faf700230fe40119169347ad72f8332d0821 to be able to promote the
RDO packages.
Change-Id: Id19998448591e9e3fb997602ec95a51cb2070889

On ubuntu these tests are flakey with ocata-m1. Let's disable them for
now.
Change-Id: Id0141edd8bf40449ef26b49ae94121f6779e0c3f

The launchpad bug was resolved+released
Change-Id: Ie20346dd6c4618866da47cded612319c03348fec

By including the whole application_catalog regex, we were trying to
run ~108 tempest tests on top of Murano.
Since Murano tempest tests already have the 'smoke' tag, let's just
rely on that to provide the testing that we need.
Murano still has 61 tests tagged as smoke but it's already a start and
we can lower that further by optimizing which tests are run directly
in the Murano tempest plugin afterwards.
Change-Id: I4d975b2c73e1f7fe8013d1ca7638a01b68a24971

The previous ec2api tempest regex was matching 161 tests so we were
trying to run all of that, greatly affecting testing duration.
We're lowering that to 14 which is much more reasonable.
Change-Id: Ib662094e588414bf7a87d2b9659d8e9ead0a0077

In specific cases, there may be conflicts between packages
provided by Centos and RDO repos. In that case RDO repos
should be used. To avoid any of these issues, this patch
updates packages right after enabling the repos.
Change-Id: Ide3265c5dc84971f19da2b5af049693cff5442b4