openstack/nova - nova - OpenDev: Free Software Needs Free Tools

This adds documentation for unified limits and signals deprecation of
the nova.quota.DbQuotaDriver.
Related to blueprint unified-limits-nova-tool-and-docs
Change-Id: I3951317111396aa4df36c5700b4d4dd33e721a74

This command aims to help migrate to unified limits quotas by reading
legacy quota limits from the Nova database and calling the Keystone API
to create corresponding unified limits.
Related to blueprint unified-limits-nova-tool-and-docs
Change-Id: I5536010ea1212918e61b3f4f22c2077fadc5ebfe

- Added function to get all attachments by instance or volume id from Cinder in Cinder.API
- Updated CinderFixture to add mock get_all_attachment functionality.
- Added unit tests get_all_attachments.
Related-Bug: 2019078
Change-Id: I8619d898f68250bf70a17b1e6b8b0c249245b43b

We leak due to running background operations like server create, rebuild
and in one case a sleeping claim. So this patch removes the leaks by
making sure that the background operations stop before the test
finishes.
As there is no more leak in functional test too this patch makes the
leak an error there too.
Change-Id: I6905999050e8d09b772837034a212c534e9c3226

The leak tests just start a server create and the finish so I think
the actual conductor thread leaks. If I add a bit of sleep to the end of
these test cases then the leak disappears. Unfortunately adding the
sleep at the greenpool fixture does not have this effect. So instead I
added delete_server calls to the end of these tests as those are nicer
than sleeps and has the same effect regarding the leak.
We are down to 3 leaking functional test
Change-Id: I070390c695283bdd9b87cd879aa2a9257ee7bdfb

This change add a global greenpool which is used to manage
the greenthreads created via nova.utils.spawn(_n).
A test fixture is also added to use an isolated greenpool
which will raise an exception if a greenthread is leaked.
the fixture will optionally raise if greenlets are leaked.
This is enabled by unit test by default and is configurable
for functional tests.
This change removes all greenthread leaks from the unit
and functional tests that were detected. 7 functional
tests still leak greenlets but they have no obvious
cause. as such greenlet leaks are not treated as errors
for funtional tests by default. Greenthread leaks
are always treated as errors.
Set NOVA_RAISE_ON_GREENLET_LEAK=1|true|yes when invoking
tox to make greenlet leaks an error for functional tests.
Change-Id: I73b4684744b340bfb80da08537a745167ddea106

Change-Id: Ib8f8bb596fa58dde40c1a0b359435ef0a48f15b2

If we have no pci_requests we will have no pci_devices, so initialize
that on create so we stop trying to lazy-load them later. Also,
migration_context will always be empty on create, so initialize that
for the same reason.
Change-Id: I546961e6018c3c48cf482cc38ca2d91a29e0da77

Right now our server show operation ends up lazy-loading the fault
and service fields from the instance instead of querying them with
the instance itself. Fault generates a failure during query if we
don't have one (which we should fix) but at least we can avoid the
services load.
Change-Id: Ib551a778438e079e19e53510cabcfa4a57202ce3

Resize ends up lazy-loading three instance properties on every run.
Rebuild loads five. We should pre-query for these so we get them
initially when we know we're going to need them.
Change-Id: I30ab3c7a147e3ea5bb633a75f6f4a8962963e43a

This makes us keep track of how many things we've lazy-loaded on a
given object instance in a single service. Ideally we'd only ever
lazy-load something zero or one times where it makes sense for
easier optimization. However, because this 'works so well' we end up
lazy-loading more things than we expect.
Change-Id: Ie379239c5de795ba8acc281da9208b3a10e92da0

The
test_get_migrations_after_live_migrate_server_in_different_project test
case took 160 sec to finish. It turned out that it start a live
migration and the never cancels it, this result in a leaked bare
greenlet. At the end of the test case the compute service is stopped and
we introduced a wait for all running greenlets at that point in
Ia916310522b007061660172fa4d63d0fde9a55ac. This wait take 160 sec to
time out.
So this change deletes the instance before the end of the test case to
cancel the live migration and avoid leaking a greenlet and make the test
case fast again.
Change-Id: I8341e328d4f5da5edc069eb69988fccb07353839

This patch concerns the time when a VM is being unshelved and the
compute manager set the task_state to spawning, claimed resources of
the VM and then called driver.spawn(). So the instance is in vm_state
SHELVED_OFFLOADED, task_state spawning.
If at this point a new update_available_resource periodic job is
started that collects all the instances assigned to the node to
calculate resource usage. However the calculation assumed that a
VM in SHELVED_OFFLOADED state does not need resource allocation on
the node (probably being removed from the node as it is offloaded)
and deleted the resource claim.
Given all this we ended up with the VM spawned successfully but having
lost the resource claim on the node.
This patch changes what we do in vm_state SHELVED_OFFLOADED, task_state
spawning. We no longer delete the resource claim in this state and
keep tracking the resource in stats.
Change-Id: I8c9944810c09d501a6d3f60f095d9817b756872d
Closes-Bug: #2025480 

Written by gibi, I just cleaned it up.
Change-Id: I8386a846b3685b8d03c59334ccfb2efbd4afe427
Co-Authored-By: Balazs Gibizer <gibizer@gmail.com>
Related-Bug: #2025480 

Debian prefers Mariadb instead of mysql, the libmysqlclient-dev pkg
is not available there, replace it with libmariadb-dev-compat.
Replace mysql-server with mariadb-server.
Change-Id: I736ecb2a18d3ca02143a41b77bd8f213d1b7ad33

Recently a change landed in devstack [1] to install packages into a
global venv by default and the "nova" command was not symlinked for
compat, so jobs using run-evacuate-hook are failing with:
 nova: command not found
We had intended to switch away from using novaclient CLI commands in our
scripts anyway, so we can just use this opportunity to switch to OSC.
[1]: If9bc7ba45522189d03f19b86cb681bb150ee2f25
Change-Id: Ifd969b84a99a9c0460bceb1a28fcee6e51cbb4ae

Change-Id: I308d6bfa7c67b8e4dab002ff2e031c34563cd842
Signed-off-by: Takashi Natsume <takanattie@gmail.com>

When the upper constraint usage moved to install_command in tox.ini, it
was not removed from docs target's deps list, where it's now not
necessary, and even could cause issues when overriding
TOX_CONSTRAINTS_FILE.
This patch replaces the old UPPER_CONSTRAINTS_FILE to the "new"
TOX_CONSTRAINTS_FILE in the install_command, too.
Change-Id: I9c27caa869d72e06e84bcf84ceedb586ed3bc8e9

This job is marked as non-voting in the parent definition, which
causes us to run it in gate as non-voting, which is just a waste of
resources. Remove it from the gate queue until it is marked as voting
in the definition (or maybe consider just leaving it out).
Depends-On: https://review.opendev.org/c/openstack/devstack/+/890136
Change-Id: I2ddbafe851be2c7826d78613f12560a1713ac1b1

The [DEFAULT] enabled_ssl_apis option is supposed to contain the subset
(or the same set) of the [DEFULT] enabled_apis option.
Because we accept only the correct values for enabled_apis, we can do
the same for enabled_ssl_apis. This would allow users to notice any
wrong values which were ignored previously.
Change-Id: Ibba8f3ea307df3e81e18c3a75361e27a0f52701d

Related to blueprint compute-object-ids
Change-Id: I6e837b9086fe20a9135712bca2d711843d39739a

Change I764b0086d0bf39d4ca8cb308696169282b12b42b removed the final use
of 'get_by_instance_uuid'. These mocks have been unused since.
Change-Id: I179720d6ed6dd2d1aca682cd1213c4d6c77ebcfe
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

The minimum required version of QEMU is now 5.2.0, and of libvirt is
7.0.0.
The below version constants get removed:
 - MIN_LIBVIRT_VIOMMU_AW_BITS
 - MIN_LIBVIRT_VDPA
 - MIN_QEMU_VDPA
 - MIN_LIBVIRT_AARCH64_CPU_COMPARE
Adjust the related unit tests accordingly.
Change-Id: Ie805eb7fa59f9f7728da27fddbd6e968e971a2e4
Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>

Based on the Libvirt and QEMU versions, two traits,
COMPUTE_ADDRESS_SPACE_PASSTHROUGH and COMPUTE_ADDRESS_SPACE_EMULATED,
are controlled. Since the two are supported from the same version on
the Libvirt and QEMU, Nova handles them in the same way.
Blueprint: libvirt-maxphysaddr-support
Depends-On: https://review.opendev.org/c/openstack/os-traits/+/871226
Signed-off-by: Nobuhiro MIKI <nmiki@yahoo-corp.jp>
Change-Id: If6c7169b7b8f43ad15a8992831824fb546e85aab