openstack/nova - nova - OpenDev: Free Software Needs Free Tools

Add a new host plugin to monitor whether the garbage collector is running.
This is then used by the wait_for_vhd_coalesce function to know whether
waiting any more is likely to produce the expected results.
Increased the number of times we loop waiting for coalesce as the loop
now has knowledge that something is still happening.
DocImpact
Change to parameter default.
Change-Id: Idcc0738945bd1aee8cdb52c9cfabd798f85c31db
Closes-bug: 1258169

The XenAPI driver does not have running CI and per the schedule,
is slated for deprecation in Icehouse-3. This adds the warning
message to the log to alert admins to this fact, in line with the
others we have been adding for drivers in similiar situations.
Change-Id: Iaff48aaca27d87c600291b8abcc23273a879366a

We are seeing this issue freqently in the gate, but we don't have any
stacktrace for what is happening. An error from libvirt is an
unexpected exception and should include a stacktrace.
Related-Bug: #1275256
Change-Id: If7c0e19aba7a3d129f4b82f52d86b8e3b6a0a194

Previous the message reflected the default value of
create_unique_mac_address_attempts configuration option.
Changed to a generic message.
Change-Id: I55c5f95cfc7ac4183e08a7880cbbb9670a02cef4
Closes-Bug: #1276312 

In the libvirt driver, we log information about the instance we're
about to generate XML for, which includes a NetworkInfo model. In
reality, this is a NetworkInfoAsyncWrapper which acquires a lock
on the model. Since we're in the middle of a log statement, we also
hold the logging lock. This happens right after we've fired off an
async request to update information about the instance, which first
acquired the network model lock and then acquired the logging lock
to make a seemingly innocuous log record.
The resulting deadlock is fixed by this patch by stringifying the
NetworkInfo object before making the log call in the libvirt
driver.
Change-Id: I81f35197ab7c74daa84fb51fbaa2df28c025da13
Closes-bug: 1276268

The baremetal driver has been accessing some libvirt driver
private classes and config variables. Any code that is intended
to be shared across virt drivers should be in a common package
instead. Add a hacking file to enforce this for imports and
config options so that no further problems like this are made
in the future.
NB, this skips verification of the baremetal driver until
bugs 1261826 and 1261827 are fixed, or the baremetal driver is
removed, whichever comes first.
Change-Id: Ifbfe597795795a847830f9bd937dc459cd37d118
Closes-Bug: #1261842 

The paths of API schema modules include "/schemas" and they show
API schema clearly. It is not necessary to include "_schema" in
module names. This patch renames the modules with removing "_schema"
from module names.
@validaiton.schema() takes 1 parameter, so calling it by name
is excessive. Then, this patch removes the specified argument.
By applying these changes, we can specify API schema module in one
line with @validation.schema().
Partially implements blueprint nova-api-validation-fw
Change-Id: Ia646e1c2466fa0a1418f757a48d2e6b4f84f7c98

With nova-BM the TFTP files (kernel and ramdisk) are cached between
instances but disks never are (today). We were treating the disk image
the same way but this is incorrect because its filename is 'disk'
rather than '$uuid' and as such it has to be cleaned out whenever a
deploy request is being processed.
Change-Id: I15c8fc25a4c064341b9f1ed51e6df8d1494fbdff
Co-Authored-By: Robert Collins <rbtcollins@hp.com>

This is needed to remove get_active_by_window from nova compute API and
allow us to make the simple_tenant_usage API use Flavor objects.
Related to blueprint compute-manager-objects
Change-Id: I93354db2775f08346f9b812f1f1e78a4050efd4a

python-keystoneclient 0.5.0 added a new oslo.config option:
 enforce_token_bind=permissive
Change-Id: If719600866561e03b037b1eb1f58e14103a99385

In line with the other drivers we've been marking as untested,
this patch marks the ESX driver as "not tested" in the logs to
alert admins to the fact. The VCenter driver is being tested
with minesweeper, and thus does not emit a similar message on
startup.
Change-Id: I9217bab01761f68b716cdce3b21c2653abd9afa1

Set the default cdrom bus to scsi when the guest architecture is
ppc64 and the hypervisor is kvm|qemu. Checks the arch from
image_meta. Fallback to host arch if arch is not present in the
metadata associated with the image.
KVM unit tests without image_meta unified in a loop.
Change-Id: I62d1490ee8c9ba45a087f934342f14386ddb7cb2

When creating a key name with characters like '/' that are not
being encoded or cannot be handled as expected by routes when
matching the url, then it's not possible to remove them anymore.
The new format apply the restriction: "[\w\.\- :]+$"
This change will fix the bug without API change.
Change-Id: I64bc098c4bde33c40fa149191a751ea3eb19f932
Closes-Bug: #1256119 

We don't need to have the vi modelines in each source file,
it can be set in a user's vimrc if required.
Also a check is added to hacking to detect if they are re-added.
Change-Id: I347307a5145b2760c69085b6ca850d6a9137ffc6
Closes-Bug: #1229324 

We will want to call attach in code paths other than booting an instance
(attaching a volume to a running instance for example). This patch
moves the boot specific log message out of the attach method.
Change-Id: Id70d3cabdd118ee1879d7b593765a5034a7ec88e

Related to blueprint nova-network-objects
Change-Id: I9139a172d9c84f13030f896fb5414cacf5877ee1

The oslo.messaging library takes the existing RPC code from oslo and
wraps it in a sane API with well defined semantics around which we can
make a commitment to retain compatibility in future.
The patch is large, but the changes can be summarized as:
 * oslo.messaging>=1.3.0a4 is required; a proper 1.3.0 release will be
 pushed before the icehouse release candidates.
 * The new rpc module has init() and cleanup() methods which manage the
 global oslo.messaging transport state. The TRANSPORT and NOTIFIER
 globals are conceptually similar to the current RPCIMPL global,
 except we're free to create and use alternate Transport objects
 in e.g. the cells code.
 * The rpc.get_{client,server,notifier}() methods are just helpers
 which wrap the global messaging state, specifiy serializers and
 specify the use of the eventlet executor.
 * In oslo.messaging, a request context is expected to be a dict so
 we add a RequestContextSerializer which can serialize to and from
 dicts using RequestContext.{to,from}_dict()
 * The allowed_rpc_exception_modules configuration option is replaced
 by an allowed_remote_exmods get_transport() parameter. This is not
 something that users ever need to configure, but it is something
 each project using oslo.messaging needs to be able to customize.
 * The nova.rpcclient module is removed; it was only a helper class
 to allow us split a lot of the more tedious changes out of this
 patch.
 * Finalizing the port from RpcProxy to RPCClient is straightforward.
 We put the default topic, version and namespace into a Target and
 contstruct the client using that.
 * Porting endpoint classes (like ComputeManager) just involves setting
 a target attribute on the class.
 * The @client_exceptions() decorator has been renamed to
 @expected_exceptions since it's used on the server side to designate
 exceptions we expect the decorated method to raise.
 * We maintain a global NOTIFIER object and create specializations of
 it with specific publisher IDs in order to avoid notification driver
 loading overhead.
 * rpc.py contains transport aliases for backwards compatibility
 purposes. setup.cfg also contains notification driver aliases for
 backwards compat.
 * The messaging options are moved about in nova.conf.sample because
 the options are advertised via a oslo.config.opts entry point and
 picked up by the generator.
 * We use messaging.ConfFixture in tests to override oslo.messaging
 config options, rather than making assumptions about the options
 registered by the library.
The porting of cells code is particularly tricky:
 * messaging.TransportURL parse() and str() replaces the
 [un]parse_transport_url() methods. Note the complication that an
 oslo.messaging transport URL can actually have multiple hosts in
 order to support message broker clustering. Also the complication
 of transport aliases in rpc.get_transport_url().
 * proxy_rpc_to_manager() is fairly nasty. Right now, we're proxying
 the on-the-wire message format over this call, but you can't supply
 such messages to oslo.messaging's cast()/call() methods. Rather than
 change the inter-cell RPC API to suit oslo.messaging, we instead
 just unpack the topic, server, method and args from the message on
 the remote side.
 cells_api.RPCClientCellsProxy is a mock RPCClient implementation
 which allows us to wrap up a RPC in the message format currently
 used for inter-cell RPCs.
 * Similarly, proxy_rpc_to_manager uses the on-the-wire format for
 exception serialization, but this format is an implementation detail
 of oslo.messaging's transport drivers. So, we need to duplicate the
 exception serialization code in cells.messaging. We may find a way
 to reconcile this in future - for example a ExceptionSerializer
 class might work, but with the current format it might be difficult
 for the deserializer to generically detect a serialized exception.
 * CellsRPCDriver.start_servers() and InterCellRPCAPI._get_client()
 need close review, but they're pretty straightforward ports of code
 to listen on some specialized topics and connect to a remote cell
 using its transport URL.
blueprint: oslo-messaging
Change-Id: Ib613e6300f2c215be90f924afbd223a3da053a69