openstack/nova - nova - OpenDev: Free Software Needs Free Tools

As noted on the mailing list some time back [1], pylint flags this as a
useless lock [2]. Make it non-useless.
[1] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/CZVC6SEMUSEH7UT5LDHOWL7WBZ2OXUWZ/
[2] https://pylint.readthedocs.io/en/latest/user_guide/messages/warning/useless-with-lock.html
Change-Id: If8243cc62c3dd9cd5f5b0d664981975efc6300cc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

As part of the Eventlet removal, this patch replaces
eventlet.semaphore.Semaphore and BoundedSemaphore with
the equivalent form threading
Change-Id: Id5edb53af765b6541e2bec4d6b9958cd0f66a862

The legacy server group member counting logic fits well to use the
existing scatter-gather logic instead of rolling its own thread
handling.
This replaces a direct eventlet dependency with an indirect, shared one,
in the scatter-gather therefore making the eventlet removal work easier
Change-Id: I6d1b5f9654df2a93bd3722a5813d5ad3a7d1c94a

OpenStack recently dropped python3.9 support for global requirements[1]
as it was removed from the Flamingo supported runtimes[2].
So this patch removes 3.9 support from nova too.
[1]https://review.opendev.org/c/openstack/requirements/+/948285
[2]https://governance.openstack.org/tc/reference/runtimes/2025.2.html#python
Change-Id: I8aea971d7972959c32d5175926cbaddb21839f8e

We have 4 ways to enter nova code and got monkey patched:
* nova.cmd - used by all of our CLI commands and non WSGI services
* nova.api.openstack - used by our WSGI services
* nova.test - used by our unit test environment to run nova services in GreenThreads.
* nova.tests.functional - used by our functional test environment to run nova services in GreenThreads.
The latter is unnecessary as all our functional test uses the nova.test
module so it automatically got monkey patches by that. So this patch
removing the monkey patching from nova.tests.functional. I don't see any
test runtime increase locally after the change so I don't think the
tests start to run more serially due to some missed monkey patch.
Change-Id: I4731dab89e2c1f1707d322c575ab0780bff80535

This change separates the evetlet monkey patching
from importing the module and add a module level
constant to track if we have already monkey patched.
Change-Id: Ic4ab0ba7a8320a008d6e246641446446dcc9ccc0

The nova debuger functionality was intended
to help debugging running process however it has
never been reliable due to our use of eventlet and is generally
not required when not using eventlet. I.e. you can just
run the nova console-scripts form a debugger or add pdb
statements as required.
As part of the eventlet removal the debugger functionality is
removed given its untested and undocumented.
Change-Id: I7bf88f06f3d1dbd2c7e342b27a21440a123c631d

Add a new hacking check to prevent adding Eventlet concurrency
primitives where the stdlib one is equivalent.
Right now only eventlet/greenthread.sleep is checked but as we progress
with the transformation we will add more primitives.
Change-Id: I1310bb7a8c8133364060eb8814226cab52219171

As part of the Eventlet removal, this patch replaces eventlet.sleep
with the equivalent time.sleep, which should work the same
with Python threads
Change-Id: I31b1aa854d8c95e47ba476051a650937b739a52b

This makes us use the new method if available, and if not, fall back
to the old method.
Change-Id: If52ac05a02b69476bd2cfa74a7ee800c3f6eeb20

As [1] removed the possibility to use the Eventlet based API servers
this patch can clean up the configuration options from the [wsgi]
section that are only used by that code path.
The remaining two options [wsgi]api_paste_config and
[wsgi]secure_proxy_ssl_header are still in use by the WSGI application
code path.
[1]I79b725f3b3569e9c1460a93ac40ca92269e7d003
Change-Id: Ia113daabab399e8db8edb1a2402ccae6fca351d5

We learned during recent installer development that triggering
GMR with apache/mod_wsgi API services via signals is hard due
to multiple reasons. We ended up using file based triggers instead of
signals. This patch document this approach.
Change-Id: I1fdbe6314ce4a1b173d01d3ebd9db07a0beb25a2

The previous patch[1] removed the entry points. As there is sizable
amount of doc change needed to remove all the references from the doc
to the removed entry points a separate patch, this, is created to do so.
[1] Ie758550c0b8fb02aeb398396961467d9f845fcc9
Change-Id: Ibe8e45e86912e747f07e5fabd5b1204341c1e606

The previous patch[1] removed the Eventlet based WSGI entry points, and
that code was the only real user of the in tree WSGIService and
WSGIServer classes, we can remove those too. This removes a good chunk
of eventlet dependency from our tree.
There is a catch though. The functional test env used these to start the
nova-metadata-api service. We re-implemented the fixture to use
load the wsgi app and use the wsgi intercept instead. This also showed
that while the Eventlet based API service could be reset via the
oslo.service interface the wsgi APP based API service cannot. So the
related cell caches reset testing is removed.
[1] Ie758550c0b8fb02aeb398396961467d9f845fcc9
Change-Id: I79b725f3b3569e9c1460a93ac40ca92269e7d003

Nova deprecated[1] running the API services under Eventlet in the Rocky
release 6 years ago. Now that we are trying to transition away from
Eventlet it is time to rip out these entry points fully.
[1] b53d81b03c
Change-Id: Ie758550c0b8fb02aeb398396961467d9f845fcc9

This change removes the concept of plug time vs
bind time live migrations events.
In past releases Id2d8d72d30075200d2b07b847c4e5568599b0d3b
and I51673e58fc8d5f051df911630f6d7a928d123a5b
added workarounds to nova to enable live migration with
the ovn backend. Over the past 5 years a lot of work has
been done in ovn and neutron to support multiple port
bindings and propagage that information to the ovn
db. As a result the workaround in nova are nolonger
required.
Related-Bug: #2073254
Change-Id: Ic3e9c93681d11d5ab988d6990e9b8d480da887d4

The request_specs_dict in the _list_view is initialized as a
defaultdict object in order to return empty string as default.
But the request_spec_dict is replaced with a normal dict object in
the v2.96 microversion, then if server list and RequestSpec missmatch
happens by any reason, the List Server API and the List
Server Detail API hit 500 Internal server error because of key error.
This commit updates the req_spec_dict to use normal dict object, then
it returns sentinel object if there is no appropriate
request_spec object.
Closes-Bug: #2095364
Change-Id: If282b8709954f276cb5d48114437809d771a9958

Related to blueprint one-time-use-devices
Change-Id: I6f764666a44c74c5ac97dced568fc685dee013b6

This adds documentation to the PCI-passthrough doc in the admin guide,
explaining how to use one-time-use devices.
Keeping this separate so we can iterate on it separate from the code.
Related to blueprint one-time-use-devices
Change-Id: Iff91c0726bbb37c7a3ef885a73e3c3586feb6004

This adds support for devices that will be allocated to an instance
once and left in a reserved=total state. An external workflow can
put them back into allocatable state by dropping reserved back to
zero. Note this requires PCI-in-placement tracking for the affected
devices and it is only valid for type-PCI and type-PF devices.
Related to blueprint one-time-use-devices
Depends-On: https://review.opendev.org/c/openstack/requirements/+/946181
Co-Authored-By: Balazs Gibizer <gibi@redhat.com>
Change-Id: Idfe8a746a97d68cd4eae30afb7d22f4e3af80327

This makes us invalidate our cache of the PCI-in-placement resource
providers when we go to do instance_claim(). This is not technically
required right now, but is setup for the next patch where we will
update that inventory during claim and we need to make sure we are
working with the latest version. Without this, we may consider a
cached version of the inventory to be the same as the proposed one,
and thus not actually update placement when we need to. Since PCI-in-
placement was designed to tolerate external changes to the inventory
(especially/explicitly changing the reserved count), we need to be
careful not to allow our cache to prevent us from taking the action
we intend.
Related to blueprint one-time-use-devices
Change-Id: I89039328af7a2d2e6a4128dd08dbe8e97ecb16cd

This makes invalidate_resource_provider() have a cacheonly flag that
only invalidates our cache, but does not remove the provider from the
tree for efficiency.
Related to blueprint one-time-use-devices
Change-Id: I04dd5e984c5671d866804c258422e4230fce37b7

This patch improves the test definitions and configurations. It also
adds mix-mode, flavor-based, and port-based integration tests."
The target goal of these series of patch is to enable VFIO devices
migration with kernel variant drivers.
Implements: blueprint migrate-vfio-devices-using-kernel-variant-drivers
Change-Id: I39f5f55bed5ddd940947b9a1e67086e85a9fe074