openstack/devstack - devstack - OpenDev: Free Software Needs Free Tools

Change-Id: I5d0697f39bab0a5ff956c3cc41c26ffe601ef6b9
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

This reverts commit 113689ee46.
Reason for revert: systemd-252-17.el9 which includes the fix is now available in CentOS 9-stream repos.
Change-Id: I6fe19838a75a30fd5d2434c03b7f403f1c7e4b50

systemd-252-16.el9 introduced a regression
where libvirtd process exits after 120s of
inactivity.
Add a workaround to unset 120s timeout for
libvirtd, the workaround can be removed once
the fix is available in systemd rpm.
Related-Bug: #2029335
Change-Id: Id6db6c17518b54d5fef7c381c509066a569aff6d

This change enables httpd in systemd so that it
starts after a reboot and updates how selinux is
disabled to use /etc/selinux/config in addtion
to setenforce.
Change-Id: I5ea8693c0b967937483bd921b1d9984ea14bc723

We haven't been testing the distro for a while in CI, e.g. in
Tempest, the jobs on opensuse15 haven't been executed for a year
now.
Therefore the patch removes opensuse support from devstack.
Closes-Bug: #2002900
Change-Id: I0f5e4c644e2d14d1b8bb5bc0096d1469febe5fcc

The job is broken since it is running with python3.7 and most services
now require at least python3.8.
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: Ie21f71acffabd78c79e2b141951ccf30a5c06445

Recent iputils release in CentOS 8-stream causing
ping failures with non root user. This needs a fix
in systemd package as mentioned in the Related Bugs,
until it's fixed and is in 8-stream mirrors let's
workaround it by setting net.ipv4.ping_group_range
setting manually.
Related-Bug: #1957941
Related-Bug: rhbz#2037807
Change-Id: I0d8dac910647968b625020c2a94e626ba5255058

openEuler is an open-source Linux based operating system. The current
openEuler kernel is based on Linux and supports multi arch, such as X86_64
and aarch64. It fully unleashes the potential of computing chips. As an
efficient, stable, and secure open-source OS built by global open-source
contributors, openEuler applies to database, big data, cloud computing,
and AI scenarios. openEuler is using RPM for package management.
Note:
Currently there is no available package for uwsgi-plugin-python3 and ovn, so that
openEuler needs manually install them from source.
Website: https://www.openeuler.org/en/
Change-Id: I169a0017998054604a63ac6c177d0f43f8a32ba6
Co-Authored-By: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>

Keystone no longer has any special functionality hidden behind the admin
site. KEYSTONE_AUTH_URI which used to point to the admin site has long
ago been changed to be a copy of KEYSTONE_SERVICE_URI, which points to
the public site.
Drop all KEYSTONE_AUTH_* variables except KEYSTONE_AUTH_URI which may
still be in use in some plugins.
This also allows to finally drop the fixup_keystone() function.
Change-Id: I549f3cadc27d137e014241cdd47e90267859c848

The official Ubuntu cloud images have some further python pkgs
preinstalled that conflict with our requirements. Allow to
overwrite them.
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Closes-Bug: 1871485
Change-Id: I793c250cae5e7b9bc835b7016d790d1f9ae8a7f3

This reverts commit 7a3a7ce876 and
bcd0acf6c0 and part of
f1ed7c77c5 which all cap our pip
installs.
Given the pip ecosystem can often incorporate major changes, tracking
upstream at least generally gives us one problem at a time to solve
rather than trying to handle version jumps when LTS distros update.
The new dependency resolver included some changes that disallow
setting URL's like "file:///path/to/project#egg=project" in
constraints. Apparently the fact it used to work was an accident of
the requires/constraints mechanism; it does make some sense as the URL
doesn't really have a version-number that the resolver can put in an
ordering graph.
The _setup_package_with_constraints_edit function comment highlights
what this is trying to do
 # Updates the constraints from REQUIREMENTS_DIR to reflect the
 # future installed state of this package. This ensures when we
 # install this package we get the from source version.
In other words; if constraints has "foo==1.2.3" and Zuul has checked
out "foo" for testing, we have to make sure pip doesn't choose version
1.2.3 from pypi.
It seems like removing the entry from upper-requirements.txt is the
important part; adding the URL path to the on-disk version was just
something that seemed to work at the time, but isn't really necessary.
We will install the package in question which will be the latest
version (from Zuul checkout) and without the package in
upper-requirements.txt nothing will try and downgrade it.
Therefore the solution proposed here is to remove the adding of the
URL parts.
This allows us to uncap pip and restore testing with the new
dependency resolver.
Closes-Bug: #1906322
Change-Id: Ib9ba52147199a9d6d0293182d5db50c4a567d677

In RHEL-based distributions, updating setuptools using pip removes the
files from the python3-setuptools RPM. It breaks some tools such as
semanage (which is used by diskimage-builder) that use the -s flag of
the python interpreter (don't import modules from /usr/local).
This commit reinstalls python3-setuptools to fix those applications.
Change-Id: Ib44857e83f75acf37823fae912960a801c83cf7f

Since victoria cycle, we have moved upstream testing to
Ubuntu Focal (20.04) and so does no Bionic distro in
Xena cycle testing runtime[1]. Grenade jobs also started
running on Focal since victoria was released.
Only thing left was legacy jobs which were not migrated to
Ubuntu Focal in Victoria and as per another community-wide
goal[2], all the lgeacy jobs were suppsoed to be migrated
to zuulv3 native jobs in victoria cycle itself. One of the
pending job was in nova (nova-grenade-multinode) which is also
migrated to zuulv3 native now
- https://review.opendev.org/c/openstack/nova/+/778885
If there is any job running on bionic, we strongly recommend
to migrate it to Ubuntu Focal.
[1] https://governance.openstack.org/tc/reference/runtimes/xena.html
[2] https://governance.openstack.org/tc/goals/selected/victoria/native-zuulv3-jobs.html
Change-Id: I39e38e4a6c2e52dd3822c9fdea354258359a9f53

This patch changes the OVN module from DevStack to allow for using the
OSapackaged version of OVN instead of compiling it from source.
A new variable called OVN_BUILD_FROM_SOURCE has been introduced and when
set to False (the default value) OVN will then use the packaged version
for setting up DevStack.
Note, in the stop_ovn() function, the OVN metadata agent service name
was wrong and the service wasn't being stopped as part of ./unstack.sh.
This patch also fixed it as well.
Change-Id: Ib41e3b486550200572afd6b3ba783d7644d70d44
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
Co-Authored-By: Slawek Kaplonski <skaplons@redhat.com>

This change updates bionic installs to use the
ussuri cloud archive to enable the use of libvirt 6.0.0.
This is required to prevent a libvirt bug that causes intermittent
failures for the tempest test_live_block_migration_paused testcase.
Change-Id: I9c395c2b5fdfe6ad9a43477280e88e9a9b34f057
Related-Bug: 1901739

workaround reported https://bugs.launchpad.net/neutron/+bug/1896945
fixed by https://review.opendev.org/#/c/755356/
Change-Id: I86a0be548e344ed4e95eab7212ba432bf570d2ae

This includes a workaround to a known dnsmasq >= 2.81 issue that
results in unanswered DHCP requests from instances as documented in the
following Neutron bug:
dnsmasq >= 2.81 not responding to DHCP requests with current q-dhcp configs
https://bugs.launchpad.net/neutron/+bug/1896945
For the time being we will attempt to downgrade to 2.80 to avoid this.
Related-Bug: #1896945
Change-Id: I3a760c43956221424926bd9dad0ebe9b28ae2b52

This is will allow the openstack/nova project to facilitate a minimum
required version bump of QEMU and libvirt within the libvirt virt driver
in I8e349849db0b1a540d295c903f1470917b82fd97 ahead of the planned switch
to focal later in Victoria.
Change-Id: I85eb45632ff229676f7c29708f4a7cc64b3d90e3

Change-Id: I2433e2ed067f866751bf49983c0a7efce4be8900

This patch fixes an early stack issue where the following error message
would be presented:
ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project
and thus we cannot accurately determine which files belong to it which
would lead to only a partial uninstall.
We also drop references to removal of Python 2 library egg infos now
that Python 2 is EOL.
Closes-Bug: #1892363
Change-Id: I2876ee58ab6b73682869d6b4e684e10ac5e56ad9

This reverts commit f6286cb586.
This patch is blocking glance as it needs mod_wsgi to perform new import workflow.
Change-Id: I4475247dfe986114d37678b3d3d552c0c7d02ddc

Those historic references to port 5000 and 35357 aren't being used
anymore for some time, so let us drop them.
Clean up some python2/3 wording along the way.
No longer mention Identity API v2, which is also a thing of the past.
Change-Id: Iafff097eee082f24ea2ae27ad038ad115aa36c61

There's a bug[1] with the combination of the p11-kit and
ca-certificates-mozilla packages available on the latest built
opensuse-15 node in nodepool (which has not been rebuilt for weeks due
to a separate issue[2]) which causes the standard CA bundle to not be
installed correctly and causes jobs that call to external HTTPS services
to fail. Upgrading both packages in sync fixes the issue.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1154871
[2] http://bugzilla.suse.com/show_bug.cgi?id=1166139
Change-Id: Ia8fdfe12fd9089e178adcb2b5eec997eebada262
Needed-by: https://review.opendev.org/713566 

This first came in with Id749c37ab7fefa96b35f11816b56b9def5ef4b08. It
talks about ancient versions of pip; can't see we need it any more.
Change-Id: I9d4831955070990a81a809d988612d9d5b1aa672

We are hitting this error:
+ tools/fixup_stuff.sh:fixup_ubuntu:82 :
 sudo rm -f /usr/lib/python3/dist-packages/httplib2-0.11.3.egg-info
rm: cannot remove
 '/usr/lib/python3/dist-packages/httplib2-0.11.3.egg-info': Is a directory
This patch adds the -r option to allow removing folders.
Change-Id: Ib7bb8b0a3dcf747bcc06da1a2fb17fa9d8808484

Following packages conflict with pip installed versions:
* httplib2
* pyasn1-modules
Change-Id: Ic4f70f839765e67394509cc543560aac7f50e287

The hack has be around for pip 1.4.1 and older. It should be safe to
remove it by now. In fact it causes problems in my Ubutu Bionic VM when
trying to overwrite httplib2 library installed from the distro package.
Change-Id: I34b826f4e8f10f8d44b888120f19fcc7ba501b3d

Need to make PyYAML overridable on Ubuntu, it is a dependency for e.g.
cloud-init, so we cannot remove it.
Depends-On: https://review.opendev.org/703792
Change-Id: I4423dfb2c30299903b52a2bb06d846dd487f5b8b

With the goals of Ussuri being Python 3.6 [1], the python 3.5
environment on Xenial is too old. Remove testing and the most obvious
bits of support from devstack.
Also drop claimed support for artful, which is long EOL.
[1] https://governance.openstack.org/tc/reference/runtimes/ussuri.html
Change-Id: Iefcca99904dde76b34efbbfc0e04515dfa5a09e5

openSUE Leap 15.0 and 15.1 both provide python3-six version 1.11.0.
Since version 1.12.0 was released, pip>=10 recognizes the version
difference and tries to uninstall the distro-provided version and fails.
This change adds another hack to remove the egg-info file for the six
library so that pip can manage it directly. We also have to wait to
install os-testr until after the fixup has happened since trying to
install it triggers the issue.
Change-Id: I4649abe06b5893a5251bfcdd4234abccde6ceda2

The existing code to disable apparmor on SUSE systems only worked for
recent openSUSE / SLE releases. On SLE12 (at least), aa-enabled and
aa-teardown are not available, so instead use systemd's interface for
stop/disable. However on newer releases, systemctl stop apparmor is a
no-op:
 https://www.suse.com/releasenotes/x86_64/SUSE-SLES/15/#fate-325343
 https://gitlab.com/apparmor/apparmor/merge_requests/81
 https://build.opensuse.org/package/view_file/openSUSE:Leap:15.2/apparmor/apparmor.service?expand=1
So we still need to call aa-teardown if it's available.
Change-Id: I8d99c8d743cc1935324e2e4fcb67efaa5241199e

This updates links going to git.openstack.org and review.openstack.org
to go to their respective opendev locations to avoid redirects.
Change-Id: I78e3bb5303718962f591117f9c0ee11f2314b128
Closes-Bug: #1833256 

The python3-ply package is indirectly a dependency of dhcp-client, which
is not exactly an optional package. Pip >=10 refuses to install ply from
global-requirements with this distro package is installed, so our only
option is to remove it manually.
Change-Id: I377fdd4a581eb4b6275584d92cafc0b783fe3b84

As noted in comments, this is a hack to get pip10 to overwrite some
package installed libraries.
Change-Id: Iea24a3ea915e13f7e0882144028ec5ff5bfdfae1

The universe repository is not enabled when installing Ubuntu from an
ISO (at least for Bionic). This leads to some errors during the devstack
run that are not seen when running based on a cloud image which has that
repo enabled by default. Enable that repository unconditionally, the
operation is idempotent.
Change-Id: Ifcb7ecd78fb25ca2136f5848c19b74500e520873
Closes-Bug: 1792936

Dnsmasq and haproxy are used frequently by neutron and nova, apparmor
profiles can block some operations and the deployed cloud can't
work properly so some tests are going to fail.
Some openSUSE distros has apparmor enabled by default so we need to
disable it.
Change-Id: I30fda684effb09810643e58bf0b31a73d7d9b378
Signed-off-by: aojeagarcia <aojeagarcia@suse.com>

This updates the UCA usage from Pike to Queens. As a result,
the various volume multiattach checks can also be removed
because the Queens UCA has libvirt 4.0.0.
Change-Id: Icb971831c8d4fe5f940d9e7993d53f1c3765e30f

Neutron functional tests want to use ubuntu cloud archive but it's
not possible to source the fixup_stuff.sh from a neutron CI setup
script. Break it up so that only the UCA portion can be executed
from neutron.
Change-Id: Ie18833bfa30f1789e63cbe9c86f5ece3453f43fb

This adds the ENABLE_VOLUME_MULTIATTACH flag and if True
configures Tempest to run volume multiattach tests.
Note that due to https://bugzilla.redhat.com/show_bug.cgi?id=1378242
we can't run multiattach tests with the Pike UCA packages since
those include qemu 2.10 and libvirt 3.6, and the valid versions for
multiattach support with libvirt is qemu<2.10 or libvirt>=3.10.
Depends-On: I80c20914c03d7371e798ca3567c37307a0d54aaa
Depends-On: I158c6f20e3e6a24bd2e5299abbeb3fc5208e5885
Part of nova blueprint multi-attach-volume
Change-Id: I46b7eabf6a28f230666f6933a087f73cb4408348

This reverts commit ee22ca8373
Depends-On: Iae2962bb86100f03fd3ad9aac3767da876291e74
Change-Id: I4d5fa052bdc5eef1795f6507589e2eaf4e093e23
Related-Bug: #1718295 

This reverts commit a7e9a5d447.
The jobs that run live migration tests are failing at about
a rate of 50% since this merged. There are no recent changes
to nova in the last 24 hours that are related to live
migration, and this is failing on the master branch only,
so I suspect the failures are due to new qemu packages
getting pulled in from this change.
Change-Id: Ic8481539c6a0cc7af08a736a625b672979435908
Closes-Bug: #1718295 

Now that Pike has been released, switch to using the pike version of
UCA instead of ocata, too.
One reason to do so it that it adds python3-ceph packages, allow to have
progress with the python3 compatibility of the Ceph integration.
Change-Id: I7d95e53892b697c72af75ad0ce7ce2dec6d31fde

In the original change I said "for infra nodes, it shouldn't do
anything anyway ...". Well that was pre-Fedora 26 :)
It seems that dnf > 2.0 now intentionally throws an error when trying
to explicitly install an ignored package. Thus, as described in the
comment, take a simpler approach of skipping this on infra nodes.
pip-and-virtualenv in dib should have installed the latest pip,
virtualenv and setuptools, so we don't want to fiddle with that
anyway.
[1] https://review.openstack.org/#/c/338998/
Change-Id: Ib300b58377a0d0fe1bd7444c71acdb9a87dc033b

Some URLs are broken, so fix them.
The others are redirect to new URLs,
so replace them with new ones.
The config options of nova serial console proxy
have been gathered in nova/conf/serial_console.py.
So the description in doc/source/guides/nova.rst
is fixed.
Change-Id: Ifd81cc09969341fbf8f135a913fc6003b94e0acc

The trove development environment is typically a linux VM within which
openstack is installed, and trove launches guest vm's within that
environment. To make it possible for these vm's to launch in one human
lifetime, one must set vt/x and enable nested hypervisors to use with
kvm; qemu emulation will take way too long.
The new libvirtd (v2.5.0) in Ubuntu Cloud Archive doesn't handle
nested hypervisors well and if you use it, you end up with a guest
hanging on the GRUB line.
To enable that use-case, we provide ENABLE_UBUNTU_CLOUD_ARCHIVE which
the trove developer can set (to False) before running devstack.
Change-Id: Ia0265c67bb7d2a438575a03c0ddbf2d9c53266ed
Closes-Bug: #1689370