openstack/openstack-ansible-rabbitmq_server - openstack-ansible-rabbitmq_server - OpenDev: Free Software Needs Free Tools

Stable feature flags must be enabled before upgrades to ensure
the service can still start. These additional tasks check for
any disabled feature flags and enable them. This is done
immediately after upgrades so that systems can run with them
enabled striaght away.
Closes-Bug: #2038818
Change-Id: I5211a30899f544a9f7e609e85551a92b245d25bf
(cherry picked from commit 5abd7b71ba)

This implements a new variable rabbitmq_custom_config to
be able to place custom configuration to the rabbitmq.conf
e.g. to configure installed plugins
Change-Id: I952eefe646b00f60184f8d353f6f055bbdc4ac90
(cherry picked from commit e46cf7c988)

Upstream has notified us[1] about the immediate deprecation and removal
of the PackageCloud repositories we are using. Thus, we must move to the
CloudSmith repos already being used for the erlang dependency for
RabbitMQ.
[1] https://github.com/rabbitmq/rabbitmq-server/discussions/8386
Closes-Bug: 2021410
Change-Id: I361757b8f76a6c2d087989a36d8f7543aba69e16
(cherry picked from commit ab9dbfb50e)

Change-Id: If8c83682093cb08e8a092550c44010134cd210d3

With this patch we ensure that duplicated records are not
created with rabbitmq role if hosts file already contain
OSA managed block. Managing hosts still might be required for
role usage outside of the OSA so we workaround this usecase.
Change-Id: Ia20902f0ffe21ce563966fee4d233e5ec3afe3d9
Related-Bug: #1960587 

Replace placing templates for systemd overrides with
systemd_service role, that will handle overrides for us in more
convenient and unified way.
Change-Id: I2759b1949e9ecc98953f414c6f9838aed7dd8499

Switch to using cloudsmith repo as a source for rabbitmq and erlang.
This allows us to be consistent about repository that is
used across supported distributions along with switching from
unreliable erlang-solutions that tend to fail periodically.
Change-Id: Ia438ee3b0aa1ba95aff014776e936516b83181ec

Starting from RabbitMQ 3.7.0 it's recommended to use new-style
config which is simply an ini file.
It's easier to read and maintain config file in ini fromat rather then
in classic erlang.
At the same time we still keep old-style config as it might have settings
that are not supported in new-style config.
There're no evidences that used there options are still supported,
but it's worth deprecating them in follow-up patch anyway.
Change-Id: I239366ad4aa2bc7a02d826b6c2f94631f4b0e622

This unifies the way of rabbitmq installation across all distros.
Also includes reno for deprecation of file installation method.
Change-Id: Idcf2d298e2808ef7b1a2160fc94cd6c1b5929182

With this configuration, the database Mnesia reduces the activity to the
disk. It is useful when a huge of queues/exchanges/bindings are created
and destroyed.
With the default value (100), RabbitMQ could log {Mnesia is overloaded}.
Moved to 300. The range should be between 100 and 1000.
Read [1] for more info.
[1] http://erlang.org/doc/man/mnesia.html#dump_log_write_threshold
Change-Id: I6dcfc9db02bcd8c8f0a1ebf58d9c3ceb84cae10a

The rabbitmq bind address can now be set using the
`rabbitmq_port_bindings` option which creates a hash of rabbitmq
connection options allowing multiple bind addresses and port
configurations to be present.
This change also organizes the config so that its now far more human
readable.
The option `rabbitmq_disable_non_tls_listeners` has been removed. This
was done because it is now irrelevant given its possible to set the
specific bind addresses and port configurations.
Change-Id: I103e406f5393a4ce3f7d6cd7f7e25e2058b0e796
Signed-off-by: Kevin Carter <kevin@cloudnull.com>

The override rabbitmq_memory_high_watermark can be used to
alter the overall memory consumption of rabbitMQ and more
importantly define when the garbage collection (gc) is happening.
The old default value of 0.4 can lead up to 80% memory usage
during gc which can lead to OOM scenarios.
The new default value is set to 0.2, lowering the maximum memory
usage to 40%
Change-Id: Iedbb459a5d17f16bbb204b0b8e989ae84c77f8a6

Without this policy, the queues reply_*, *_fanout_* are mirrored, across
all the RabbitMQ nodes.
It is not necessary to mirror the queues above, because they are usually
bound to one specific consumer, so the mirroring is not essential.
The new policy reduces the number of mirror queues and increases, in
general, the performances because it reduces the number of copies across
the cluster.
More info about RabbitMQ HA [1]
[1]: https://www.rabbitmq.com/ha.html
ref:
http://eavesdrop.openstack.org/meetings/oslo/2019/oslo.2019年02月25日-15.22.txt
Change-Id: I783d314aaa68b09abb4fed90818165b9e61e9758

We make remote network hits to get the GPG keys which are quite
unreliable, and apt_key does not support using a proxy properly [1]
so let's store them inside the role and use them.
The implementation here matches that which was done in the
galera_client role in I520ccbadf3320b0d07fc83e3dbec9ea2bd16ec83
[1] https://github.com/ansible/ansible/issues/31691
Change-Id: I2715c904975b7940af72bd422904e748d3bae953

The version of erlang we're using, while functional, is not recommended.
This change updates our version of erlang for rabbitmq to be the
recommended version by changing the pinned version of erlang.
The apt package pinning meta dependency has been removed and put into
the apt task file as an include role. To ensure a more uniform process
for setting the version of erlang used with rabbitmq the new variable
`rabbitmq_erlang_version_spec` has been added. This option has a default
of "null" and is set by distro specific variables when applicable.
Erlang version support matrix: https://www.rabbitmq.com/which-erlang.html
Change-Id: Iced12d4533eec068bd11d8bd235d81308ef40427
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

This patch optimizes the Erlang/RabbitMQ installation on CentOS. It
includes:
 * Installing RabbitMQ from RabbitMQ's repository
 * Installing Erlang all-in-one from RabbitMQ's repository
 * Remove old versionlocks before applying new ones (fixes bug)
The erlang-solutions repository is hosted in eastern Europe and has
high latency to the USA and western Europe. Installing from that
repository brings in over 80 individual packages, which causes
additional delays in the role.
The Erlang all-in-one package from RabbitMQ's repository contains
only the Erlang bits that RabbitMQ needs. Also, it has HiPE enabled
by default, which is recommended by RabbitMQ developers for
better performance.
Closes-Bug: 1712596
Change-Id: I1bfcc96f353bd27b0004d93e250bb041eee48bdb

The release note didn't make it in with the RabbitMQ ulimit
adjustment patch. This patch adds the note.
Closes-Bug: 1714511
Change-Id: I858f459fad829571c66f3baac4432c3d0318820d

The RabbitMQ community and project recommend erlang 19.3.x for the best,
most stable, results. This change downloads the recommended version of
erlang [ https://www.rabbitmq.com/which-erlang.html ] from ESL and moves
the erlang packages for distros into a specific list installed only when
the ``rabbitmq_install_method`` variable is not set to "file".
Change-Id: Ief57b1fdcb791f0f8ed20c1cbb54ea19d4373f81
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Deployers can override the `rabbitmq_disable_non_tls_listeners`
variable, setting a value of `True` if they wish to enable this feature.
Change-Id: I4fe39099dbe8973d2655845c19882c404d4f20b1

In some cases, deployers may want to use a repo instead of a deb
file (limited connectivity, repo mgmt tools already in place, ...)
This commit introduces this behavior in an optional manner: The
default behavior is still using a deb file. If using an external
repository, the repo used will be RabbitMQ recommended one hosted
on packagecloud.io.
Change-Id: If366d7411cffd59e4988ceba6fc47eac2b384ddf

Previously the role assumed that all deployers wanted
all queues to be HA. Performance testing by Mirantis and
Intel has suggested that throughput increases can be dramatic
when disabling HA queues, see:
 http://docs.openstack.org/developer/performance-docs/test_results/mq/rabbitmq/index.html
Therefore, we should allow deployers to set their own
policies when desired.
Change-Id: I156b8618e78d74823ec568053157afc853baad84
Closes-Bug: #1607830 

The ``rabbitmq_server`` role now supports configuring HiPE compilation
of the RabbitMQ server Erlang code. This configuration option may improve
server performance for some workloads and hardware. Deployers can
override the ``rabbitmq_hipe_compile`` variable, setting a value of
``True`` if they wish to enable this feature.
Change-Id: I433d94eff00ac82a9069f9092faa87d449190442

The current method of installing the distribution packages required is
set in the tasks and cannot be changed by a deployer.
Currently the apt task always installs the latest package. This results
in unexpected binary changes when a deployer may simply be trying to
execute a configuration change.
This patch adds the ability for a deployer to change the desired state
so that the results are predictable.
Change-Id: I3227dc6f15e0307926e65427bc635c34e2baa87f

The default value of "rabbitmq_all" remains, but this
change should allow deployers to target the role to multiple
groups now.
Change-Id: Idc1bc44b005f09f9dc7eb9778719a8ba92756c06
Implements: blueprint multi-rabbitmq-clusters

Separate files have been created for vars and tasks related to a
specific package manager. The 'vars_files_var' variable has been created
to store a list of files to search for distro specific variables.
The 'rabbitmq_apt_packages' variable has been deprecated and renamed to
the more generalized 'rabbitmq_dependencies' to better describe its
purpose and to simplify reuse of existing install tasks between multiple
distros.
Change-Id: I1940593978b733501daf5fe25edd393f2f6bee0c

Change-Id: I6683e5227bf1b9f438efc2612f84311acfbd7b51