Files
864f4d1ef05e3e539d3537fa4c8d731df9571c9d
devstack /lib /rpc_backend

194 lines
6.5 KiB
Plaintext
Raw Normal View History

#!/bin/bash
#
# lib/rpc_backend
# Interface for installing RabbitMQ on the system
# Dependencies:
#
# - ``functions`` file
# - ``RABBIT_{HOST|PASSWORD|USERID}`` must be defined when RabbitMQ is used
# ``stack.sh`` calls the entry points in this order:
#
# - check_rpc_backend
# - install_rpc_backend
# - restart_rpc_backend
# - iniset_rpc_backend (stable interface)
#
# Note: if implementing an out of tree plugin for an RPC backend, you
# should install all services through normal plugin methods, then
# redefine ``iniset_rpc_backend`` in your code. That's the one portion
# of this file which is a standard interface.
# Save trace setting
Namespace XTRACE commands I noticed this when debugging some grenade issues failures. An include of grenade/functions stores the current value of XTRACE (on) and disables xtrace for the rest of the import. We then include devstack's "functions" library, which now overwrites the stored value of XTRACE the current state; i.e. disabled. When it finishes it restores the prior state (disabled), and then grenade restores the same value of XTRACE (disabled). The result is that xtrace is incorrectly disabled until the next time it just happens to be turned on. The solution is to name-space the store of the current-value of xtrace so when we finish sourcing a file, we always restore the tracing value to what it was when we entered. Some files had already discovered this. In general there is inconsistency around the setting of the variable, and a lot of obvious copy-paste. This brings consistency across all files by using _XTRACE_* prefixes for the sotre/restore of tracing values. Change-Id: Iba7739eada5711d9c269cb4127fa712e9f961695
2015年10月13日 11:03:03 +11:00
_XTRACE_RPC_BACKEND=$(set +o | grep xtrace)
set +o xtrace
RABBIT_USERID=${RABBIT_USERID:-stackrabbit}
if is_service_enabled rabbit; then
RABBIT_HOST=${RABBIT_HOST:-$SERVICE_HOST}
fi
# Functions
# ---------
# clean up after rpc backend - eradicate all traces so changing backends
# produces a clean switch
function cleanup_rpc_backend {
if is_service_enabled rabbit; then
# Obliterate rabbitmq-server
uninstall_package rabbitmq-server
# in case it's not actually running, /bin/true at the end
sudo killall epmd || sudo killall -9 epmd || /bin/true
if is_ubuntu; then
# And the Erlang runtime too
apt_get purge -y erlang*
fi
Add an option to enable version 1.0 of the AMQP messaging protocol This change adds the RPC_MESSAGING_PROTOCOL configuration option that selects the messaging protocol that is used by the RPC backend and client. Some brokers can support different kinds of 'on the wire' messaging protocols. Qpid, for example, supports both AMQP 0-10 (the default), and AMQP 1.0. Use the RPC_MESSAGING_PROTOCOL configuration variable to override the default protocol for those brokers that support multiple protocol options. This new option is necessary in order to enable the new AMQP 1.0 oslo.messaging transport as described in the blueprint. Note well: currently this AMQP 1.0 functionality is only available on fedora 19+ platforms. Support is WIP on ubuntu/debian and rhel/centos 7. Enabling the RPC_MESSAGING_PROTOCOL option on an unsupported platform will cause devstack to exit with an approriate error message. Change-Id: Ib8dea59922844e87d6c947b5dca557f5b5fc1160 Implements: blueprint amqp10-driver-implementation
2014年07月23日 16:44:37 -04:00
fi
}
# install rpc backend
function install_rpc_backend {
if is_service_enabled rabbit; then
# Install rabbitmq-server
install_package rabbitmq-server
if is_suse; then
install_package rabbitmq-server-plugins
# the default systemd socket activation only listens on the loopback interface
# which causes rabbitmq to try to start its own epmd
sudo mkdir -p /etc/systemd/system/epmd.socket.d
cat <<EOF | sudo tee /etc/systemd/system/epmd.socket.d/ports.conf >/dev/null
[Socket]
ListenStream=
ListenStream=[::]:4369
EOF
sudo systemctl daemon-reload
sudo systemctl restart epmd.socket epmd.service
fi
if is_fedora || is_suse; then
# NOTE(jangutter): If rabbitmq is not running (as in a fresh
# install) then rabbit_setuser triggers epmd@0.0.0.0.socket with
# socket activation. This fails the first time and does not get
# cleared. It is benign, but the workaround is to start rabbitmq a
# bit earlier for RPM based distros.
sudo systemctl --now enable rabbitmq-server
fi
fi
}
# restart the rpc backend
function restart_rpc_backend {
if is_service_enabled rabbit; then
# Start rabbitmq-server
echo_summary "Starting RabbitMQ"
# NOTE(bnemec): Retry initial rabbitmq configuration to deal with
# the fact that sometimes it fails to start properly.
Fix rabbitmq retry for error checking I think this retry check has been broken since we introduced "set -e". Unfortunately it seems the issue of rabbitmq not starting first-time persists on centos 7 hosts occasionally, e.g. [1]: --- + rabbit_setuser stackrabbit secretrabbit + local user=stackrabbit pass=secretrabbit found= out= ++ sudo rabbitmqctl list_users Error: unable to connect to node 'rabbit@devstack-centos7-rax-iad-100675': nodedown DIAGNOSTICS =========== nodes in question: ['rabbit@devstack-centos7-rax-iad-100675'] hosts, their running nodes and ports: - devstack-centos7-rax-iad-100675: [{rabbitmqctl29293,39511}] current node details: - node name: 'rabbitmqctl29293@devstack-centos7-rax-iad-100675' - home dir: /var/lib/rabbitmq - cookie hash: KieJnx1pnllKbHVihGcDqA== --- Fix up this retry while we investigate [2] [1] http://logs.openstack.org/64/141864/1/check//check-tempest-dsvm-centos7/4308f0c/logs/devstacklog.txt.gz [2] https://bugzilla.redhat.com/show_bug.cgi?id=1144100 Change-Id: I11fb3728e08adc1e0f7acca63e5a308d24dce78e
2014年12月16日 09:53:36 +11:00
# Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1144100
# NOTE(tonyb): Extend the original retry logic to only restart rabbitmq
Change the restart_rpc_backend loop to accomodate async rabbitmq Some distros have converted to systemd for starting RabbitMQ. This has resulted in: --- [Call Trace] ./stack.sh:904:restart_rpc_backend /home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201:die [ERROR] /home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201 Failed to set rabbitmq password Error on exit World dumping... see /opt/stack/logs/worlddump-2015年05月29日-031618.txt for details --- Because 'restart_service rabbitmq-server' returns before the server is ready to accept connections. Alter the retry loop to only restart the rabbitmq-server every second time through the loop. Allowing time for the slow rabbit to start. Closes-Bug: 1449056 Change-Id: Ibb291c1ecfd109f9ed10b5f194933364985cc1ce
2015年05月15日 12:51:43 +10:00
# every second time around the loop.
# See: https://bugs.launchpad.net/devstack/+bug/1449056 for details on
# why this is needed. This can bee seen on vivid and Debian unstable
# (May 2015)
# TODO(tonyb): Remove this when Debian and Ubuntu have a fixed systemd
# service file.
local i
Change the restart_rpc_backend loop to accomodate async rabbitmq Some distros have converted to systemd for starting RabbitMQ. This has resulted in: --- [Call Trace] ./stack.sh:904:restart_rpc_backend /home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201:die [ERROR] /home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201 Failed to set rabbitmq password Error on exit World dumping... see /opt/stack/logs/worlddump-2015年05月29日-031618.txt for details --- Because 'restart_service rabbitmq-server' returns before the server is ready to accept connections. Alter the retry loop to only restart the rabbitmq-server every second time through the loop. Allowing time for the slow rabbit to start. Closes-Bug: 1449056 Change-Id: Ibb291c1ecfd109f9ed10b5f194933364985cc1ce
2015年05月15日 12:51:43 +10:00
for i in `seq 20`; do
Fix rabbitmq retry for error checking I think this retry check has been broken since we introduced "set -e". Unfortunately it seems the issue of rabbitmq not starting first-time persists on centos 7 hosts occasionally, e.g. [1]: --- + rabbit_setuser stackrabbit secretrabbit + local user=stackrabbit pass=secretrabbit found= out= ++ sudo rabbitmqctl list_users Error: unable to connect to node 'rabbit@devstack-centos7-rax-iad-100675': nodedown DIAGNOSTICS =========== nodes in question: ['rabbit@devstack-centos7-rax-iad-100675'] hosts, their running nodes and ports: - devstack-centos7-rax-iad-100675: [{rabbitmqctl29293,39511}] current node details: - node name: 'rabbitmqctl29293@devstack-centos7-rax-iad-100675' - home dir: /var/lib/rabbitmq - cookie hash: KieJnx1pnllKbHVihGcDqA== --- Fix up this retry while we investigate [2] [1] http://logs.openstack.org/64/141864/1/check//check-tempest-dsvm-centos7/4308f0c/logs/devstacklog.txt.gz [2] https://bugzilla.redhat.com/show_bug.cgi?id=1144100 Change-Id: I11fb3728e08adc1e0f7acca63e5a308d24dce78e
2014年12月16日 09:53:36 +11:00
local rc=0
Change the restart_rpc_backend loop to accomodate async rabbitmq Some distros have converted to systemd for starting RabbitMQ. This has resulted in: --- [Call Trace] ./stack.sh:904:restart_rpc_backend /home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201:die [ERROR] /home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201 Failed to set rabbitmq password Error on exit World dumping... see /opt/stack/logs/worlddump-2015年05月29日-031618.txt for details --- Because 'restart_service rabbitmq-server' returns before the server is ready to accept connections. Alter the retry loop to only restart the rabbitmq-server every second time through the loop. Allowing time for the slow rabbit to start. Closes-Bug: 1449056 Change-Id: Ibb291c1ecfd109f9ed10b5f194933364985cc1ce
2015年05月15日 12:51:43 +10:00
[[ $i -eq "20" ]] && die $LINENO "Failed to set rabbitmq password"
Fix rabbitmq retry for error checking I think this retry check has been broken since we introduced "set -e". Unfortunately it seems the issue of rabbitmq not starting first-time persists on centos 7 hosts occasionally, e.g. [1]: --- + rabbit_setuser stackrabbit secretrabbit + local user=stackrabbit pass=secretrabbit found= out= ++ sudo rabbitmqctl list_users Error: unable to connect to node 'rabbit@devstack-centos7-rax-iad-100675': nodedown DIAGNOSTICS =========== nodes in question: ['rabbit@devstack-centos7-rax-iad-100675'] hosts, their running nodes and ports: - devstack-centos7-rax-iad-100675: [{rabbitmqctl29293,39511}] current node details: - node name: 'rabbitmqctl29293@devstack-centos7-rax-iad-100675' - home dir: /var/lib/rabbitmq - cookie hash: KieJnx1pnllKbHVihGcDqA== --- Fix up this retry while we investigate [2] [1] http://logs.openstack.org/64/141864/1/check//check-tempest-dsvm-centos7/4308f0c/logs/devstacklog.txt.gz [2] https://bugzilla.redhat.com/show_bug.cgi?id=1144100 Change-Id: I11fb3728e08adc1e0f7acca63e5a308d24dce78e
2014年12月16日 09:53:36 +11:00
Change the restart_rpc_backend loop to accomodate async rabbitmq Some distros have converted to systemd for starting RabbitMQ. This has resulted in: --- [Call Trace] ./stack.sh:904:restart_rpc_backend /home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201:die [ERROR] /home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201 Failed to set rabbitmq password Error on exit World dumping... see /opt/stack/logs/worlddump-2015年05月29日-031618.txt for details --- Because 'restart_service rabbitmq-server' returns before the server is ready to accept connections. Alter the retry loop to only restart the rabbitmq-server every second time through the loop. Allowing time for the slow rabbit to start. Closes-Bug: 1449056 Change-Id: Ibb291c1ecfd109f9ed10b5f194933364985cc1ce
2015年05月15日 12:51:43 +10:00
if [[ $(( i % 2 )) == "0" ]] ; then
restart_service rabbitmq-server
fi
Fix rabbitmq retry for error checking I think this retry check has been broken since we introduced "set -e". Unfortunately it seems the issue of rabbitmq not starting first-time persists on centos 7 hosts occasionally, e.g. [1]: --- + rabbit_setuser stackrabbit secretrabbit + local user=stackrabbit pass=secretrabbit found= out= ++ sudo rabbitmqctl list_users Error: unable to connect to node 'rabbit@devstack-centos7-rax-iad-100675': nodedown DIAGNOSTICS =========== nodes in question: ['rabbit@devstack-centos7-rax-iad-100675'] hosts, their running nodes and ports: - devstack-centos7-rax-iad-100675: [{rabbitmqctl29293,39511}] current node details: - node name: 'rabbitmqctl29293@devstack-centos7-rax-iad-100675' - home dir: /var/lib/rabbitmq - cookie hash: KieJnx1pnllKbHVihGcDqA== --- Fix up this retry while we investigate [2] [1] http://logs.openstack.org/64/141864/1/check//check-tempest-dsvm-centos7/4308f0c/logs/devstacklog.txt.gz [2] https://bugzilla.redhat.com/show_bug.cgi?id=1144100 Change-Id: I11fb3728e08adc1e0f7acca63e5a308d24dce78e
2014年12月16日 09:53:36 +11:00
rabbit_setuser "$RABBIT_USERID" "$RABBIT_PASSWORD" || rc=$?
if [ $rc -ne 0 ]; then
continue
fi
# change the rabbit password since the default is "guest"
Fix rabbitmq retry for error checking I think this retry check has been broken since we introduced "set -e". Unfortunately it seems the issue of rabbitmq not starting first-time persists on centos 7 hosts occasionally, e.g. [1]: --- + rabbit_setuser stackrabbit secretrabbit + local user=stackrabbit pass=secretrabbit found= out= ++ sudo rabbitmqctl list_users Error: unable to connect to node 'rabbit@devstack-centos7-rax-iad-100675': nodedown DIAGNOSTICS =========== nodes in question: ['rabbit@devstack-centos7-rax-iad-100675'] hosts, their running nodes and ports: - devstack-centos7-rax-iad-100675: [{rabbitmqctl29293,39511}] current node details: - node name: 'rabbitmqctl29293@devstack-centos7-rax-iad-100675' - home dir: /var/lib/rabbitmq - cookie hash: KieJnx1pnllKbHVihGcDqA== --- Fix up this retry while we investigate [2] [1] http://logs.openstack.org/64/141864/1/check//check-tempest-dsvm-centos7/4308f0c/logs/devstacklog.txt.gz [2] https://bugzilla.redhat.com/show_bug.cgi?id=1144100 Change-Id: I11fb3728e08adc1e0f7acca63e5a308d24dce78e
2014年12月16日 09:53:36 +11:00
sudo rabbitmqctl change_password \
$RABBIT_USERID $RABBIT_PASSWORD || rc=$?
if [ $rc -ne 0 ]; then
continue;
fi
break
done
# NOTE(frickler): Remove the default guest user
sudo rabbitmqctl delete_user guest || true
fi
}
# adds a vhost to the rpc backend
function rpc_backend_add_vhost {
local vhost="1ドル"
if is_service_enabled rabbit; then
if [ -z `sudo rabbitmqctl list_vhosts | grep $vhost` ]; then
sudo rabbitmqctl add_vhost $vhost
sudo rabbitmqctl set_permissions -p $vhost $RABBIT_USERID ".*" ".*" ".*"
fi
else
echo 'RPC backend does not support vhosts'
return 1
fi
}
# Returns the address of the RPC backend in URL format.
function get_transport_url {
local virtual_host=1ドル
if is_service_enabled rabbit || { [ -n "$RABBIT_HOST" ] && [ -n "$RABBIT_PASSWORD" ]; }; then
echo "rabbit://$RABBIT_USERID:$RABBIT_PASSWORD@$RABBIT_HOST:5672/$virtual_host"
fi
}
# Returns the address of the Notification backend in URL format. This
# should be used to set the transport_url option in the
# oslo_messaging_notifications group.
function get_notification_url {
local virtual_host=1ドル
if is_service_enabled rabbit || { [ -n "$RABBIT_HOST" ] && [ -n "$RABBIT_PASSWORD" ]; }; then
echo "rabbit://$RABBIT_USERID:$RABBIT_PASSWORD@$RABBIT_HOST:5672/$virtual_host"
fi
}
# iniset configuration
function iniset_rpc_backend {
local package=1ドル
local file=2ドル
local section=${3:-DEFAULT}
local virtual_host=4ドル
if is_service_enabled rabbit || { [ -n "$RABBIT_HOST" ] && [ -n "$RABBIT_PASSWORD" ]; }; then
iniset $file $section transport_url $(get_transport_url "$virtual_host")
if [ -n "$RABBIT_HEARTBEAT_TIMEOUT_THRESHOLD" ]; then
iniset $file oslo_messaging_rabbit heartbeat_timeout_threshold $RABBIT_HEARTBEAT_TIMEOUT_THRESHOLD
fi
if [ -n "$RABBIT_HEARTBEAT_RATE" ]; then
iniset $file oslo_messaging_rabbit heartbeat_rate $RABBIT_HEARTBEAT_RATE
fi
fi
}
function rabbit_setuser {
local user="1ドル" pass="2ドル" found="" out=""
out=$(sudo rabbitmqctl list_users) ||
{ echo "failed to list users" 1>&2; return 1; }
found=$(echo "$out" | awk '1ドル == user { print 1ドル }' "user=$user")
if [ "$found" = "$user" ]; then
sudo rabbitmqctl change_password "$user" "$pass" ||
{ echo "failed changing pass for '$user'" 1>&2; return 1; }
else
sudo rabbitmqctl add_user "$user" "$pass" ||
{ echo "failed changing pass for $user"; return 1; }
fi
sudo rabbitmqctl set_permissions "$user" ".*" ".*" ".*"
}
# Restore xtrace
Namespace XTRACE commands I noticed this when debugging some grenade issues failures. An include of grenade/functions stores the current value of XTRACE (on) and disables xtrace for the rest of the import. We then include devstack's "functions" library, which now overwrites the stored value of XTRACE the current state; i.e. disabled. When it finishes it restores the prior state (disabled), and then grenade restores the same value of XTRACE (disabled). The result is that xtrace is incorrectly disabled until the next time it just happens to be turned on. The solution is to name-space the store of the current-value of xtrace so when we finish sourcing a file, we always restore the tracing value to what it was when we entered. Some files had already discovered this. In general there is inconsistency around the setting of the variable, and a lot of obvious copy-paste. This brings consistency across all files by using _XTRACE_* prefixes for the sotre/restore of tracing values. Change-Id: Iba7739eada5711d9c269cb4127fa712e9f961695
2015年10月13日 11:03:03 +11:00
$_XTRACE_RPC_BACKEND
# Tell emacs to use shell-script-mode
## Local variables:
## mode: shell-script
## End: