openstack/nova - nova - OpenDev: Free Software Needs Free Tools

Change-Id: I9bf33343610630c7c1d1a75ab3a8509f78a3231e

Change-Id: I00bea6e97e95b8aa7d3881f7c21ac6dcccc78a27

Change-Id: Ic319ea3d4c4065f63db176b53382f9b98a4e17a9

Change-Id: Ia0470265fabcbfeac762252aeec2c399540da07b

Change-Id: Ia7fe0408883476b40dd0667fd6f56f5d068ca03f

Change-Id: Ibf423f14a5c37aa298b2115bfd4936f660c6f530

Hard code pbr to d2to1 to >=0.2.10,<0.3 and
pbr to >=0.5,<0.6 for consistency across projects.
Change-Id: I7163bef37934856e5f80df336044f8ebcb690797
Signed-off-by: Chuck Short <chuck.short@canonical.com>

Declarative approach to packaging, obviates the need for copying files.
Also, removes the need for setup.py to import files from the nova
module.
./run_tests.sh now ensure's that nova.egg-info/entry_points.txt is up to
date before running tests.
Change-Id: I7b7a18e065a62491ade54fbed5d8400db1d4e6c7

As part of the move of plugins to entrypoints, take advantage of the
entrypoints based console scripts, which will make our command line scripts
available for unittesting.
Part of blueprint entrypoints-plugins
Co-authored-by: Michael Still <mikal@stillhq.com>
Change-Id: I5f17348b7b3cc896c92263dd518abb128757d81f

Open Havana development by setting version to 2013.2.
Change-Id: I37917d28a1f9e0adc2fe3e382412ebc4ed0f3bee

The cfg API is now available via the oslo-config library, so switch to
it and remove the copied-and-pasted version.
Add the 2013.1b4 tarball to tools/pip-requires - this will be changed
to 'oslo-config>=2013.1' when oslo-config is published to pypi. This
will happen in time for grizzly final.
Add dependency_links to setup.py so that oslo-config can be installed
from the tarball URL specified in pip-requires.
Remove the 'deps = pep8==1.3.3' from tox.ini as it means all the other
deps get installed with easy_install which can't install oslo-config
from the URL.
Make tools/hacking.py include oslo in IMPORT_EXCEPTIONS like it already
does for paste. It turns out imp.find_module() doesn't correct handle
namespace packages.
Retain dummy cfg.py file until keystoneclient middleware has been
updated (I18c450174277c8e2d15ed93879da6cd92074c27a).
Change-Id: I4815aeb8a9341a31a250e920157f15ee15cfc5bc

Based on https://review.openstack.org/#/c/19957/
Change-Id: Ieefee2af8812d0217e95eeffb3cf53d8e55c8fe6

Add nova-spicehtml5proxy which provides a websockets proxy,
equivalent to nova-novncproxy
Blueprint: libvirt-spice
Change-Id: I48be78c97bb7dd6635fd4bba476ef22701418ba1
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

Part 3 of 6: blueprint general-bare-metal-provisioning-framework.
Change-Id: Ia19ce00edb84aa924c2ab2c9c2217f6b49073d69
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>

a part of blueprint general-bare-metal-provisioning-framework.
Implement nova-baremetal-deploy-helper.
This service listens for HTTP requests from baremetal deploy ramdisk,
formats the remote disk and writes an image to it, as part of baremetal
PXE provisioning.
blueprint improve-baremetal-pxe-deploy shows how we plan to improve this
process.
Change-Id: I0a1b020cc5f81d49559acd4dcc781397a58e2c01
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>

This introduces *EXPERIMENTAL* compute cells functionality as a way to
scale nova in a more distributed fashion without having to use complicated
technologies like DB and message queue clustering.
Cells are configured as a tree and the top level cell should contain
nova-api without any nova-computes while child cells contain everything
except nova-api. One can think of a cell as a normal nova deployment in
that each cell has its own DB server and message queue broker.
The top level cell keeps a subset of data about ALL instances in all
cells in its DB. Child cells send messages to the top level cell when
instances change state. Data in 1 child cell is not shared with another
child cell.
A new service, nova-cells, is introduced that handles communication
between cells and picking of a cell for new instances. This service is
required for every cell. Communication between cells is pluggable with
the only option currently implemented being communnication via RPC.
Cells scheduling is separate from host scheduling. nova-cells first picks
a cell (currently randomly -- future patches add filtering/weighing
functionality and decisions can be based on broadcasts of
capacity/capabilities). Once a cell has been selected and the new build
request has reached its nova-cells service, it'll be sent over to the host
scheduler in that cell and the build proceeds as it does without cells.
New config options are introduced for enabling and configuring the cells
code. Cells is disabled by default. All of the config options below go
under a '[cells]' section in nova.conf. These are the options that one
may want to tweak:
enable -- Turn on cells code (default is False)
name -- Name of the current cell.
capabilities -- List of arbitrary key=value pairs defining capabilities
 of the current cell. These are sent to parent cells,
 but aren't used in scheduling until later filter/weight
 support is added.
call_timeout -- How long to wait for replies from a calls between cells
When using cells, the compute API class must be changed in the API cell,
so that requests can be proxied via nova-cells down to the correct cell
properly. Thus, config requirements for API cell:
--
[DEFAULT]
compute_api_class=nova.compute.cells_api.ComputeCellsAPI.
[cells]
enable=True
name=api-cell
--
Config requirements for child cell:
--
[cells]
enable=True
name=child-cell1
--
Another requirement is populating the 'cells' DB table in each cell.
Each cell needs to know about its parent and children and how to
communicate with them (message broker location, credentials, etc).
Implements blueprint nova-compute-cells
DocImpact
Change-Id: I1b52788ea9d7753365d175abf39bdbc22ba822fe

Add bin/nova-conductor to setup.py.
Change-Id: Iaa6d8625c4db0a15bb0e04e4499f2bfa2931c145
Signed-off-by: Chuck Short <chuck.short@canonical.com>

This removes the majority of the nova-volume code from the codebase.
It updates relevent config options to default to cinder. It updates a
number of existing tests that were depending on code that was removed.
A few things still need to be removed:
 * volume/driver.py & volume/iscsi.py
 These files are used by the libvirt volume driver tests. These
 tests should be updated to mock the relevant calls.
 * scheduler/simple.py & scheduler/multi.py
 These files should no longer be necessary so they can be removed
 in a subsequent patch
 * exception.py cleanup
 Once the above files are removed there are a number of unused
 exceptions which can be removed
 * database calls and database tables
 The database calls have not been removed and the tables have not
 been dropped. This can be done in a separate migration
 * additional config options and nova.conf.sample
 There may be a few extra config options that can be removed and the
 conf sample can be regenerated
Implements bp delete-nova-volume
Change-Id: I0b540e54dbabd26901a7530035a38583bb521fda

Then glob and os modules are not being used and no longer need to be
imported.
Change-Id: I0d9862811338fe6c8014c2c50c78c2cca721f9d0

Add classifiers so we can eventually register the project
on PyPI to reserve the name, even though we won't release
packages there.
Change-Id: I35f4fd72de7781d3e01f1d122d5d3267544ffbd6
Signed-off-by: Doug Hellmann <doug.hellmann@dreamhost.com>

Updates setup.py to set install_requires using parse_requirements from
openstack-common. This allows DevStack to install nova requirements, as
currently they are not handled.
Also relaxes the version requirement for lxml to >=2.3,<3. This allows
DevStack to install nova requirements without needing to install lxml
over the installed python-xml system package, currently 2.3.2.
Change-Id: Iccebf9604ba5c6fcacb4c5fa5004f9fcb08944f3

find_data_files was copied in a while back to work around setuptools
deficiencies. It has since made its way into distribute, and we are using it
from there - thus do not need a copy in our tree anymore.
Change-Id: I8bc76a201c325571601ac8da292ebd5690668065

The instance usage audit cronjob that generates periodic
compute.instance.exists notifications is not particularly scalable.
It is run on one server and takes longer as the number of instances grows.
This change moves the generation of those events to a periodic task in
the compute manager. It also adds an api extension that can be used
by administrators to check for errors generating these events.
Change-Id: I856d3d0c73c34e570112f1345d306308ef20a9ae

This gets us up to date with common/setup.py and replaces custom nova
autodoc generation with the port of that code found in common.
Change-Id: I2a1c5d2c0fdcf40dbea50cc123b537adb068cdc2

Instead of a hard-coded MANIFEST.in to list non-code files to be included in
the tarball, use the setuptools-git plugin to ensure that every file that's
in git will wind up in the tarball. The only things needed in MANIFEST.in
now are files that are generated as part of the sdist run.
Change-Id: I76df685d58f7b56977d859e8df92e845f450b3e4

Fix setup.py to include bin/nova-novncproxy in generated tarballs.
Fixes bug 1018829.
Change-Id: Iad2aa32602224e15bb879f81a13b764fc09d9c67

Change-Id: I7d941a97bf250428e70e8a4ee65ab6cb70fdfcfb

Fixes LP Bug #1008625.
Change-Id: I2b0dc97133b3039818a52e0b8456dc31cdbd714e

Fix setup.py to include a reference to bin/volume-usage-audit
and make it included in generated tarballs. Fixes bug #1001218.
Change-Id: I9a02e3723a29f3fb960ffa4b10d5499e98bd3a4c

Ubuntu uses a tarball to generate packages for Nova and
other openstack projects. This allows the user to find out
what is included in the tarball.
Signed-off-by: Chuck Short <chuck.short@canonical.com>
Change-Id: I0291e19eee1ff2c5fc98b499571b2563841c6076

blueprint remove-nova-direct-api
Change-Id: I3229f8d7f37d66fcd6b978966f3a428a69e08bb1

Broken and outdated (last updated on Mon Jun 27 2011)
Change-Id: I3ed67430e01c2db531b89bc3ffd6e7b174791fbc

* Remove core vsa code (api, manager, drivers)
* Remove nova-vsa executable
* Remove OpenStack Compute API v2 vsa extension
* Remove vsa scheduler
* Remove vsa db api methods
* Remove Zadara volume driver
* Do not migrate out any existing data
* Fixes bug 954490
Change-Id: Idab3d60796d5edbc23ef9f0887fcc1af558c6215

Change-Id: Ia67f35374917d7037887b2d2c39ca92ff79587c8

Commit 5702f80a0 updated local_BuildDoc's parameters without
also updating references to the original parameters. This fixes
the issue and gets './setup.py build_sphinx' working again.
Change-Id: I3483c906c13347a718e1794786830f06af700ebd

Looks like this fixes all HACKING problems that were around.
Thanks to Dina Belova and Alexander Kovalev for this work.
Change-Id: I8157f0d4890184c1216aab63ef7180ee8b7a184d

If we structure things like this, we can get things
actually, you know, installed.
Locales ast and oc aren't supported by babel. We'll re-import them
from Launchpad once we get babel updated.
If you want translations installed, run:
 python setup.py compile_catalog
Before you run either setup.py sdist or setup.py install
More work is needed to actually properly use the installed translations, but
we're closer.
Change-Id: I4a4bedd982c89063aa09688c9cbcf97054bcb34b

We moved to DistUtilsExtra because it promised a more automatic workflow
from setup.py. It doesn't actually deliver though, and it also vomits
warnings during setup.py usage, and also breaks pip integration.
So this is going back to babel. Mostly folks shouldn't need to know
anything about this. We'll need to update the translations import
jobs in Jenkins, and we might need to add a corresponding translations
upload job that runs post-merge.
Translations installation doesn't fully work - but it actually wasn't fully
working before. Getting this part of the project done now though is a
pre-requisite for using tox for multi-python testing (tox starts by
creating an sdist tarball and then installing it into the venv, which
is just 100% broken with DistUtilsExtra)
Change-Id: I126e1bcfab0656eab6ca10de67d3d2aaa8b844f3

Removes copy of ajaxterm code, nova-ajax-console-proxy,
and support for get_ajax_console from Nova proper.
Implements blueprint remove-ajaxterm
Fixes bug 917963
Change-Id: I2c0ff427c53c0f63a18b10475d6b4cbe9a085d83

* Adds new worker for cert management
 * Makes decrypt use an rpc to the worker
 * Moves CA filesystem creation out of cloud.setup
 * Moves test for X509 into crypto
 * Adds test for encrypting and decrypting using cert
 * Cleans up extra code in cloudpipe
 * Fixes bug 918563
 * Prepares for a future patch that will fix bug 903345
Change-Id: I4693c50c8f432706f97395af39e736f49d60e719

* Creates a unified way to access vnc consoles for xenserver and libvirt
 * Now supports both java and websocket clients
 * Removes nova-vncproxy - a replacement version of this (nova-novncproxy) can be found as described in vncconsole.rst
 * Adds nova-xvpvncproxy, which supports a java vnc client
 * Adds api extension to access java and novnc access_urls
 * Fixes proxy server to close/shutdown sockets more cleanly
 * Address style feedback
 * Use new-style extension format
 * Fix setup.py
 * utils.gen_uuid must be wrapped like str(utils.gen_uuid()) or it can't be serialized
Change-Id: I5e42e2f160e8e3476269bd64b0e8aa77e66c918c

If distutils-extra is installed, all scripts are automatically included
in the tarball. We should explicitly include them in setup.py to avoid
confusion.
As an example, the 2011.3 tarballs appear to have been generated with
distutils-extra but the 2011年3月1日/2012.1 tarballs are now being
generated in a venv. So, these newer tarballs are missing scripts which
were included in 2011.3.
Change-Id: I2a85418b84b93e041f6c7d31833b09acdcb29daa

Change-Id: I6759e5b6250c48cc0deb4b198b44c948c64c47d1

* Moves openstack/v2 directory to compute and fixes tests accordingly
 * Moves some code from api/openstack/compute to shared location, for use by volume api
 * Implements basic volume functionality for types, volumes, and snapshots
 * Changes service name from osapi to osapi_compute (and adds osapi_volume)
 * Renames nova-api-os to nova-api-os-compute, adds nove-api-os-volume
 * Separate extension mechanism for compute and volume
 ** Removes flag osapi_extension and replaces with osapi_compute_extension and osapi_volume_extension
 * Updates the paste config
 * Fixes setup.py to include nova-os-api-compute and nova-os-api-volume
 * Fix bug in volume version code that occurred as result of trunk merge
 * Update integrated/test_volumes.py to use new endpoint
Change-Id: I4c2e57c3cafd4e1a9e2ff3ce201c8cf28326afcd

Loading install_requires with the contents of pip-requires
isn't getting us any real beneift and is causing issues.
a) It can conflict with installing nova into an environment
 where deps have been installed from packages (devstack)
b) It breaks the ability to use -e git urls in pip-requires
 which we want to start using for python-novaclient and
 python-keystoneclient
c) It causes spurious network traffic when we're trying to
 test things.
At the same time, since we are not expecting anyone to
install nova from setup.py for production, the normal benefit
of the feature is not needed.
Change-Id: I402f975c652dc1c2b54b0c362b8abbb6886f8844

Change-Id: I6e372152946dc7b51a4d84d6f4b893774e7d1a1d

Alternative to using a sudoers file to limit which commands can be run as
root in Nova. This one makes use of command filters defined in Nova code
itself, which can be customized to deeply inspect command arguments before
allowing a command to be executed.
This change puts the infrastructure in place, together with command filters
that replicate the level of filtering provided by a sudoers file (no deep
argument inspection yet). An example of an advanced filter (RegExpFilter) is
also provided. This new root wrapper is not active by default (root_helper
still defaults to "sudo"). Implements blueprint nova-rootwrap.
Change-Id: I7ad723b55e9446758876f21b4fbb09374a910425