openstack/devstack - devstack - OpenDev: Free Software Needs Free Tools

This is an initial pass at plugin infrastructure for devstack which
allows specifying an external repository via:
enable_plugin <name> <giturl> [branch]
It implements the devstack specification for this at
I173dee3d57967b1d2ffd30e4868a2832aeac97ce
Change-Id: I8e4175313b3cf0b12e981122358b1288a7eb0746

This has been deleted from the nova tree, we should purge it from
devstack, as it will not work any more.
Change-Id: I20501fec140998b91c9ddfd84b7b10168624430a

Now that we are on pip 1.5.6 lets drop the workaround to make pip 1.4
work. As this is a development/testing tool requiring a newer pip
shouldn't be an issue. Also stack.sh installs pip by default.
Work around introduced in https://github.com/pypa/pip/issues/709
Change-Id: I0e7aad1d21f4fce4c020ce36685bb56893c66bdc

`stop_dstat` function was extracted into $TOP_DIR/lib/dstat in commit
e0b08d04ab but the new lib was not sourced in
unstack.sh.
Change-Id: I6878552d9f384fdd7882e52000b37ac6a60844c3
Closes-Bug: 1366713

In order to use dstat on the new side of grenade, pull dstat code into
its own lib with the function start_dstat
Change-Id: I5c908d594a6f3a90ed4b3f744002bf606841cf07

Certificates and the index were left in the data directory after
running unstack. That would break devstack on the next run.
partial blueprint devstack-https
Change-Id: I6eb552a76fb29addf4d02254c027e473d6805df1

The bm_poseur git repository link has been broken
for over 11 months. The virtualized/fake baremetal
environment is not working and has not worked in a
long time. Now, on the tail of enabling 'enable -o errexit',
this functionality now has a hard break.
Change-Id: I3cbd8db58c422bc5273d2433278aaa5e449ecfd9
Closes-Bug: 1285954

This converts the special cases in the is_service_enabled() function to call
individual functions declared by the projects. This allows projects that
are not in the DevStack repo and called via the extras.d plugin to handle
an equivalent service alias.
* Ceilometer
* Cinder
* Glance
* Neutron
* Nova
* Swift
TODO: remove the tests from is_service_enabled() after a transition period
Patch Set 2: Rebased
Change-Id: Ic78be433f93a9dd5f46be548bdbd4c984e0da6e7

Also adds an is_ironic_enabled() function to prepare for an upcoming
change in is_service_enabled().
Change-Id: I6e6e0e8b70221e231785ab27e9b5d4836933ac4c

Also adds an is_trove_enabled() function to match
https://review.openstack.org/69497 changes for is_service_enabled().
Change-Id: Ic0408ff6d9816aec8a3506931470470342a5dcd7

unstack.sh and stack.sh both have to "configure projects", but the
code was different. This change makes it so the 2 sections of the
files are the same.
Change-Id: Ia06f8bbfbe2a6e87fb406e34e13a39bd7fa9e5af

- add lib/ceilometer and lib/heat to source list for when stop_heat
and stop_ceilometer functions are called.
- add lib/tls source to lib/keystone for when is_ssl_enabled_service
function called.
Change-Id: Ief05766e9cfda71fb6392c8a757d04751283414e
Closes-Bug: #1269047 

* Save PID when using screen in screen_it()
* Add screen_stop()
* Call out service stop_*() in unstack.sh functions so screen_stop()
 can do its thing
Closes-bug: 1183449
Change-Id: Iac84231cfda960c4197de5b6e8ba6eb19225169a

Add hooks to stack.sh, unstack.sh and clean.sh to call the extras.d
scripts at multiple points in stack.sh. This allows these scripts to
perform installation and startup tasks at similar times as they would
if integrated into stack.sh.
extras.d/70-tempest.sh is present as an example of the structure
of these scripts.
See extras.d/README.md for more information.
Change-Id: Ic1fe522559b94d204d6c0319a2e3d23684c8d028

Automatically creating a new user account is not always the right course
of action when stack.sh is running as root. Plus, the re-exec did not
work correctly in some cases.
* Create tools/create-stack-user.sh to set up a suitable user
 for running DevStack
* Abort stack.sh and unstack.sh if running as root and suggest creating a
 suitable user account.
Change-Id: I5d967c00c89f32e861449234ea8fe19261cd9ae3

- Added changes to stackrc for the Trove Repos.
- Added support to devstack for "install", "configure",
 "init", and "run"
implements blueprint:trove-devstack-integration
Change-Id: Ib3f6daad33e629f764a174b80762c808ce8588e2

* Add basic support for hypervisor plugins in lib/nova_plugins
* Add lib/nova_plugins/hypervisor-docker to use Docker as a Nova
 hypervisor.
* Add tools/install_docker.sh to install the Docker daemon and
 registry container, download base image and import
* Configure Nova to use docker plugin
* Add docker exercise and skip unsupported ones
Nova blueprint: new-hypervisor-docker
Change-Id: I9e7065b562dce2ce853def583ab1165886612227

Ironic is an OpenStack project than brings a
separate service for baremetal provisioning.
Currently Ironic is in incubation but it needs
to have basic support in devstack to provide
automatic deployment testing.
Change-Id: Ide65a1379fa207a6c8b2c7d9a4f9c874b10fd9ba

1.install apache and wsgi module
2.config apache2 vhost and wsgi files for proxy,
account, container and object server.
3.refactor apache functions from horizon and swift
into lib/apache
Change-Id: I3a5d1e511c5dca1e6d01a1adca8fda0a43d4f632
Implements: blueprint enable-apache-frontend-for-swift

There is a bug in pip [1] where it will choose to install a package
from an existing build-dir if it exists over the version actually
requested.
Thus if a prior component has installed a later version of the
package, the unpacked code is already in /tmp/$USER-pip-build; it gets
re-installed and manifests in a confusing error along the lines of
---
 Downloading/unpacking requests>=1.1,<1.2.3
 (from -r /home/stack//python-cinderclient/requirements.txt (line 5))
 Running setup.py egg_info for package requests
 Requested requests>=1.1,<1.2.3 (from -r
 /home/stack/python-cinderclient/requirements.txt (line 5)),
 but installing version 1.2.3
...
 error: Installed distribution requests 1.2.3 conflicts with
 requirement requests>=1.1,<1.2.3
---
I believe pip 1.4 fixes this problem, but it should always be safe to
specify a unique build-directory for pip installs to avoid picking up
old versions.
We also add a cleanup_tmp function for clearing out anything that
stack.sh might leave around when un-stacking, and add a catch-all for
the pip-build dir.
[1] https://github.com/pypa/pip/issues/709
Change-Id: I7ce919cddfd6d6175ae67bd864f82e256ebc7090

Note: Nova and Horizon are not updated until those projects have
migrated.
Change-Id: I256ef20e7caadd9c96e6dd908c5d8b69ca5c4aeb

Also removes OVS ports
Fix Bug #1156837
Change-Id: Id4178c0a12a3ada76403a57a736a9c7a908d6a4a

- Set by default SWIFT_REPLICAS to 1 since this is the most common use
 case when using it in devstack.
- If we have swift_replicas to 1 launch the object container account and
 proxy servers in foreground in screen.
- Allow any s- services if we have just 'swift' in service enabled
 instead of having to specify all s- services to enable swift. This will be
 removed in the future.
- Set object server starting at 6013 instead of 6010 to not conflict
 with remote ssh x11 forwarding.
Change-Id: I890b6953b70283bfa0927fff0cf5e92f3c08455b

This reverts commit 5a5cbf7274.
This breaks on a default openstack install on Ubuntu 12.10,
revert for now until this can be cleaned up to work out of the
box.
Change-Id: I185509cc30bd28e920cdab60fc92129949bd6b0d

- Set by default SWIFT_REPLICAS to 1 since this is the most common use
 case when using it in devstack.
- If we have swift_replicas to 1 launch the object container account and
 proxy servers in foreground in screen.
- Allow any s- services if we have just 'swift' in service enabled
 instead of having to specify all s- services to enable swift. This will be
 removed in the future.
Change-Id: I496f79e14f99bd7e9f2c7deee12a4b6e935c3a5b

- Fixes bug 1049553.
Change-Id: I9fef93d25512c014dfb882adf0e169487bf877d8

This adds an extras.d directory to contain startup scripts that
stack.sh runs automatically at the end. Similar to local.sh
except the scripts are sourced into the stack.sh process rather
than executed as a child process. This gives them complete
access to the stack.sh environment.
Convert Tempest to use this format as an example.
Change-Id: Ibc95e6aaecf4211da948319eb452293ae4357780

Use bm_poseur to create VM and network bridge
so that, in the absence of physical hardware,
baremetal driver still has something to manipulate.
Change-Id: Id80ede13a35e4380f358b47f08d41ff98ea9d70f

* Move quantum installation to lib/quantum
* Refactor quantum configuration
* Move Quantum service account creation from keystone_data.sh to lib/quantum
* Define generic functions to install third party programs
* Minor cleanups related to Quantum
 * Kill dnsmasq which watches an interface 'ns-XXXXXX' in unstack.sh
 * Set default_floating_pool in nova.conf to make default flaoting pool
 work when PUBLIC_NETWORK_NAME is other than 'nova'
 * Make tempest work even when PRIVATE_NETWORK_NAME is other than 'private'
Change-Id: I4a6e7fcebfb11556968f53ab6a0e862ce16bb139

cinder currently has issues that leave volumes around
after tempest tests. Make sure that cinder gets cleaned
up to a zero state on an unstack.sh so that we can reset
the environment.
Change-Id: I448340899bf0fae7d4d16fa26da17feafcef888f

* Adds lib/tls to create test CA/certs
* Start proxy if 'tls-proxy' is enabled
* Configure keystone service catalog for TLS
* Tear down proxy in unstack.sh
* Set auth protocol and ca-cert chain in openrc
* Add DATA_DIR to stackrc
This is the first in a series of patches to enable TLS support
for the service API endpoints.
Change-Id: Ia1c91dc8f1aaf94fbec9dc71da322559a83d14b6

sudo is only allowed in stack.sh on the CI, so move setup_quantum code
to the stack.sh.
also fixes quantum debug command setup for linuxbridge and ryu
Change-Id: I11bc0aa242a690e25acc088b3e9f483ceab38f26

This replaces all of the [[ "$os_PACKAGE" = "deb" ]] tests, except when
those tests are before straight calls to dpkg.
Change-Id: I8a3ebf1b1bc5a55d736f9258d5ba1d24dabf04ea

* Optimize loops
* Move install steps to the lib/swift
Change-Id: Ie8a74b2627395620ccb0501171fa0150ee7497f2

pre holiday refactor extrodinare, get the horizon code
over fully into lib/horizon so that all these fixes aren't
scattered through stack.sh
Change-Id: I7f26c5c6708d5693048eb7b1ce792122adbc7351

* nova-volume has been removed in Grizzly
* part of delete-nova-volume
Change-Id: Iba91d69950767823d77aaaa93243b0f476dbb04d

This patch adds an interface for supporting multiple database backend
types and implemnts support for PostgreSQL. It also adds a function,
use_exclusive_service, which serves as a base for enabling a service
that conflicts with other services. The use_database function uses it,
and it might also be useful for selecting messaging backends.
MySQL is still selected by default. Tested on Fedora 17 and Ubuntu
12.04 with MySQL and PostgreSQL. Implements blueprint postgresql-support
Change-Id: I4b1373e25676fd9a9809fe70cb4a6450a2479174

This patch allows using the Quantum Ryu plugin.
Ryu plugin lets Quantum link Open vSwitch and Ryu OpenFlow controller[1].
Ryu OpenFlow controller is not Openstack component, but I added some
processing that is related with Ryu to stack.sh for the convenience of
the person who intend to try the plugin.
Instructions for using Ryu plugin:
1. Enable services: "q-svc", "q-agt", "q-dhcp", "q-l3", "quantum", "ryu"
2. Set Q_PLUGIN to "ryu"
3. Set an internal network interface name to connect br-int on plural
 hosts to RYU_INTERNAL_INTERFACE (optional)
Example localrc:
 disable_service n-net
 enable_service q-svc q-agt q-dhcp q-l3 quantum ryu
 Q_PLUGIN=ryu
 RYU_INTERNAL_INTERFACE=eth1
[1] http://osrg.github.com/ryu/
Change-Id: Ic1da132fa421f1c70c10a319ee3239831b0f956f

in quantum dhcp agent section
Change-Id: I602df7875710336125cdbb8407ff293a8e7d448e

*Check that tgt driver is in good state when we start
 if not try to just restart it
 *Remove targets based on iqn entries in persist files
 *Remove the persist files themeselves
 *Stop the tgt service
Updated to use *_STATE_DIR vars
Change-Id: I3ba9f2b8c099c7f290696760527fe2147d9b1d38

Fixes bug 978205.
Change-Id: I688887e4b4426db36438f2bf0e537956f4f94757

* functions
* stack.sh
* stackrc
* unstack.sh
A recent commit to stack.sh broke the RST formatting done by shocco to
produce the HTML-formatted files on devstack.org. A bunch of comment
and spacing fixes were done (ala pep8 if there were such a thing for
shell scripts).
The only non-comment changes made were to the content of some error
messages.
Fixes bug 1042271
Change-Id: Id1c74cf25c03c4f18ed741f8026e36b0d4a598dd

- We have moved swift to keystone middleware to swift use it.
- Should be referenced now as keystoneauth.
- Make swift-init silently pass in unstack even when swift was not yet
 installed.
Change-Id: Ibf9443608b7aa81be5f48f555e95ff5f4c8065bd

On non-deb systems, 'tgt' is 'tgtd'.
Change-Id: I357b47cf117a5e615eb4af9603b7c5670e5cff1c

If the Quantum dhcp-agent is enabled then we ensure that the dnsmasq
process is termated when running unstack.sh
Change-Id: I0aee8b806c61dff42400a3b8552b7c748e9c5adf

* whitespace and comment cleanups only
Change-Id: I31b3057657bb3bc5c4e085886ef6ac561967706f