openstack/openstack-ansible-rabbitmq_server - openstack-ansible-rabbitmq_server - OpenDev: Free Software Needs Free Tools

Upstream has notified us[1] about the immediate deprecation and removal
of the PackageCloud repositories we are using. Thus, we must move to the
CloudSmith repos already being used for the erlang dependency for
RabbitMQ.
[1] https://github.com/rabbitmq/rabbitmq-server/discussions/8386
Closes-Bug: 2021410
Change-Id: I361757b8f76a6c2d087989a36d8f7543aba69e16

Change-Id: If8c83682093cb08e8a092550c44010134cd210d3

Add file to the reno documentation build to show release notes for
stable/zed.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/zed.
Sem-Ver: feature
Change-Id: Ic392f8ee059787620d67be181e998b0484ef443b

With this patch we ensure that duplicated records are not
created with rabbitmq role if hosts file already contain
OSA managed block. Managing hosts still might be required for
role usage outside of the OSA so we workaround this usecase.
Change-Id: Ia20902f0ffe21ce563966fee4d233e5ec3afe3d9
Related-Bug: #1960587 

Replace placing templates for systemd overrides with
systemd_service role, that will handle overrides for us in more
convenient and unified way.
Change-Id: I2759b1949e9ecc98953f414c6f9838aed7dd8499

Switch to using cloudsmith repo as a source for rabbitmq and erlang.
This allows us to be consistent about repository that is
used across supported distributions along with switching from
unreliable erlang-solutions that tend to fail periodically.
Change-Id: Ia438ee3b0aa1ba95aff014776e936516b83181ec

Starting from RabbitMQ 3.7.0 it's recommended to use new-style
config which is simply an ini file.
It's easier to read and maintain config file in ini fromat rather then
in classic erlang.
At the same time we still keep old-style config as it might have settings
that are not supported in new-style config.
There're no evidences that used there options are still supported,
but it's worth deprecating them in follow-up patch anyway.
Change-Id: I239366ad4aa2bc7a02d826b6c2f94631f4b0e622

The sync from https://review.opendev.org/733244 updated to
openstackdocstheme 2.2.1 and reno 3.1.0 versions.
Set openstackdocs_pdf_link to link to PDF file. Note that
the link to the published document only works on docs.openstack.org
where the PDF file is placed in the top-level html directory. The
site-preview places the PDF in a pdf directory.
openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.
See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html
Change-Id: I5c6c99763cffd75824c31efd37caec6e4828ac43

Add file to the reno documentation build to show release notes for
stable/ussuri.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/ussuri.
Change-Id: I80561fc623a89b89d6c2bc1143c751d543863d4e
Sem-Ver: feature

New version of openstackdocstheme (Victoria+) respects pygments_style.
Since this repo is using now Victoria (master) requirements but has
not branched for Ussuri yet, it uses the new version.
Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.
Change-Id: Iad5aa6453e61bf08555805072e9b6e433e656e93

This repo is now testing only with Python 3, so let's make
a few cleanups:
- Remove setup.* files, those are not needed for this repo.
- Cleanup */source/conf.py to remove now obsolete content.
- Remove install_command from tox.ini, the default is fine,
 move constraints into deps
- Enable warnings for docs build
Change-Id: I07da37d6f15aefc7b2fa558a6ead7028d6c9c556

This unifies the way of rabbitmq installation across all distros.
Also includes reno for deprecation of file installation method.
Change-Id: Idcf2d298e2808ef7b1a2160fc94cd6c1b5929182

This patch replaces git.openstack.org with opendev.org as redirection
from old path was enabled.
Change-Id: Iae8d1f21e85f73efb56230cffc3ad39793ec3f3b

Add file to the reno documentation build to show release notes for
stable/train.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/train.
Change-Id: I56250e12e8eb596af69c69ad847a4c068e9abf68
Sem-Ver: feature

Add file to the reno documentation build to show release notes for
stable/stein.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/stein.
Change-Id: I2962c47e99404ed6ea0edea96744f6119c988a52
Sem-Ver: feature

With this configuration, the database Mnesia reduces the activity to the
disk. It is useful when a huge of queues/exchanges/bindings are created
and destroyed.
With the default value (100), RabbitMQ could log {Mnesia is overloaded}.
Moved to 300. The range should be between 100 and 1000.
Read [1] for more info.
[1] http://erlang.org/doc/man/mnesia.html#dump_log_write_threshold
Change-Id: I6dcfc9db02bcd8c8f0a1ebf58d9c3ceb84cae10a

The rabbitmq bind address can now be set using the
`rabbitmq_port_bindings` option which creates a hash of rabbitmq
connection options allowing multiple bind addresses and port
configurations to be present.
This change also organizes the config so that its now far more human
readable.
The option `rabbitmq_disable_non_tls_listeners` has been removed. This
was done because it is now irrelevant given its possible to set the
specific bind addresses and port configurations.
Change-Id: I103e406f5393a4ce3f7d6cd7f7e25e2058b0e796
Signed-off-by: Kevin Carter <kevin@cloudnull.com>

The override rabbitmq_memory_high_watermark can be used to
alter the overall memory consumption of rabbitMQ and more
importantly define when the garbage collection (gc) is happening.
The old default value of 0.4 can lead up to 80% memory usage
during gc which can lead to OOM scenarios.
The new default value is set to 0.2, lowering the maximum memory
usage to 40%
Change-Id: Iedbb459a5d17f16bbb204b0b8e989ae84c77f8a6

Without this policy, the queues reply_*, *_fanout_* are mirrored, across
all the RabbitMQ nodes.
It is not necessary to mirror the queues above, because they are usually
bound to one specific consumer, so the mirroring is not essential.
The new policy reduces the number of mirror queues and increases, in
general, the performances because it reduces the number of copies across
the cluster.
More info about RabbitMQ HA [1]
[1]: https://www.rabbitmq.com/ha.html
ref:
http://eavesdrop.openstack.org/meetings/oslo/2019/oslo.2019年02月25日-15.22.txt
Change-Id: I783d314aaa68b09abb4fed90818165b9e61e9758

We make remote network hits to get the GPG keys which are quite
unreliable, and apt_key does not support using a proxy properly [1]
so let's store them inside the role and use them.
The implementation here matches that which was done in the
galera_client role in I520ccbadf3320b0d07fc83e3dbec9ea2bd16ec83
[1] https://github.com/ansible/ansible/issues/31691
Change-Id: I2715c904975b7940af72bd422904e748d3bae953

Change-Id: Ib0a55b2907baf5fd8f37c02b3538edb02729ab1d

The version of erlang we're using, while functional, is not recommended.
This change updates our version of erlang for rabbitmq to be the
recommended version by changing the pinned version of erlang.
The apt package pinning meta dependency has been removed and put into
the apt task file as an include role. To ensure a more uniform process
for setting the version of erlang used with rabbitmq the new variable
`rabbitmq_erlang_version_spec` has been added. This option has a default
of "null" and is set by distro specific variables when applicable.
Erlang version support matrix: https://www.rabbitmq.com/which-erlang.html
Change-Id: Iced12d4533eec068bd11d8bd235d81308ef40427
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Change-Id: I957251be9dce419a6db8a272986fc5d12bfe3e9b

Release notes are version independent, so remove version/release
values. We've found that projects now require the service package
to be installed in order to build release notes, and this is entirely
due to the current convention of pulling in the version information.
Release notes should not need installation in order to build, so this
unnecessary version setting needs to be removed.
This is needed for new release notes publishing, see
I56909152975f731a9d2c21b2825b972195e48ee8 and the discussion starting
at
http://lists.openstack.org/pipermail/openstack-dev/2017-November/124480.html
.
Change-Id: I467712d9122270788f19d2fc8aa9481b1379bcf0

This patch implements an initial set of jobs intended to match
the current job execution method. It does not intend to improve
how the jobs are executed - only to replicate what is currently
in openstack-infra/openstack-zuul-jobs and provide the platform
to iterate on.
Change-Id: I7a1c312268b7219cb3e04058e9563a2fe0c469f0

This patch optimizes the Erlang/RabbitMQ installation on CentOS. It
includes:
 * Installing RabbitMQ from RabbitMQ's repository
 * Installing Erlang all-in-one from RabbitMQ's repository
 * Remove old versionlocks before applying new ones (fixes bug)
The erlang-solutions repository is hosted in eastern Europe and has
high latency to the USA and western Europe. Installing from that
repository brings in over 80 individual packages, which causes
additional delays in the role.
The Erlang all-in-one package from RabbitMQ's repository contains
only the Erlang bits that RabbitMQ needs. Also, it has HiPE enabled
by default, which is recommended by RabbitMQ developers for
better performance.
Closes-Bug: 1712596
Change-Id: I1bfcc96f353bd27b0004d93e250bb041eee48bdb

The release note didn't make it in with the RabbitMQ ulimit
adjustment patch. This patch adds the note.
Closes-Bug: 1714511
Change-Id: I858f459fad829571c66f3baac4432c3d0318820d

Change-Id: I1c5e7a626ff386eccfc603946233601f72df1c7d

To use openstackdocstheme 1.11.0 properly, this patch fixes
some settings according to follow[0].
[0]https://docs.openstack.org/openstackdocstheme/latest/
Change-Id: Ib2d216eeef4c78e0f9ff24d8b8e531fa2ad833c1

The RabbitMQ community and project recommend erlang 19.3.x for the best,
most stable, results. This change downloads the recommended version of
erlang [ https://www.rabbitmq.com/which-erlang.html ] from ESL and moves
the erlang packages for distros into a specific list installed only when
the ``rabbitmq_install_method`` variable is not set to "file".
Change-Id: Ief57b1fdcb791f0f8ed20c1cbb54ea19d4373f81
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Trivialfix
Change-Id: I96d954332991745dcbaf6d5f0ec4ef50c41aba72

Change-Id: Idd9f0ace812af9c4a244b35c3d8cb2fe71e2bc52

Deployers can override the `rabbitmq_disable_non_tls_listeners`
variable, setting a value of `True` if they wish to enable this feature.
Change-Id: I4fe39099dbe8973d2655845c19882c404d4f20b1

In some cases, deployers may want to use a repo instead of a deb
file (limited connectivity, repo mgmt tools already in place, ...)
This commit introduces this behavior in an optional manner: The
default behavior is still using a deb file. If using an external
repository, the repo used will be RabbitMQ recommended one hosted
on packagecloud.io.
Change-Id: If366d7411cffd59e4988ceba6fc47eac2b384ddf

Releasenote translation publishing is being prepared. 'locale_dirs'
needs to be defined in conf.py to generate translated version of the
release notes.
Note that this repository might not get translated release notes - or
no translations at all - but we add the entry here nevertheless to
prepare for it.
Change-Id: I33f512560c3299527897998d1a8250549af259ff

Change-Id: Iccb5da23204729d57671ab746202f8df3d54e106

Previously the role assumed that all deployers wanted
all queues to be HA. Performance testing by Mirantis and
Intel has suggested that throughput increases can be dramatic
when disabling HA queues, see:
 http://docs.openstack.org/developer/performance-docs/test_results/mq/rabbitmq/index.html
Therefore, we should allow deployers to set their own
policies when desired.
Change-Id: I156b8618e78d74823ec568053157afc853baad84
Closes-Bug: #1607830 

The ``rabbitmq_server`` role now supports configuring HiPE compilation
of the RabbitMQ server Erlang code. This configuration option may improve
server performance for some workloads and hardware. Deployers can
override the ``rabbitmq_hipe_compile`` variable, setting a value of
``True`` if they wish to enable this feature.
Change-Id: I433d94eff00ac82a9069f9092faa87d449190442

The current method of installing the distribution packages required is
set in the tasks and cannot be changed by a deployer.
Currently the apt task always installs the latest package. This results
in unexpected binary changes when a deployer may simply be trying to
execute a configuration change.
This patch adds the ability for a deployer to change the desired state
so that the results are predictable.
Change-Id: I3227dc6f15e0307926e65427bc635c34e2baa87f

The default value of "rabbitmq_all" remains, but this
change should allow deployers to target the role to multiple
groups now.
Change-Id: Idc1bc44b005f09f9dc7eb9778719a8ba92756c06
Implements: blueprint multi-rabbitmq-clusters

This patch updates the sphinx configuration for docs and releasenotes
to make it easier to replicate across repositories and to comply with
pep8 testing without exceptions.
Change-Id: I389cf6c3490adb516cf90305393471b47f83a937

Separate files have been created for vars and tasks related to a
specific package manager. The 'vars_files_var' variable has been created
to store a list of files to search for distro specific variables.
The 'rabbitmq_apt_packages' variable has been deprecated and renamed to
the more generalized 'rabbitmq_dependencies' to better describe its
purpose and to simplify reuse of existing install tasks between multiple
distros.
Change-Id: I1940593978b733501daf5fe25edd393f2f6bee0c

Change-Id: I582a5c6d7283df438527858b3ec3f17920361431

Change-Id: I6683e5227bf1b9f438efc2612f84311acfbd7b51