openstack/openstack-ansible-rabbitmq_server - openstack-ansible-rabbitmq_server - OpenDev: Free Software Needs Free Tools

Change-Id: I377b26dab9c0939c5de701fef9ebf4d230e19701

As verification of the cluster state is generic action which needs
to be included and registered at different parts of the role, it
makes sense to include it separately and evaluate feature flag conditions
during tasks include
Change-Id: Ib00c5072a663cdccf05156badc43cab675e8de9f

We can replace shell with a simple command using json formatter,
which allows us to select only missing flags and enable them expclicitly
rather then enable all flags.
This also produces a better log, representing which flags were missing.
Change-Id: Idb5c02109458771853e0fdbc7f6bb27beaa731b4

Current behaviour with just pinning on Ubuntu/Debian leads to potential
situations, when there's no selected version available in repo to
proceed with what's available based on set priorities.
Instead, it might be worth to fail explicitly when requested version is
not available. This also aligns behaviour with EL distros.
Change-Id: Iad69cc4b5f8ca40246bcbc604e77eef7975eb104

At the moment rabbitmq_upgrade flag is being used on unhealthy
clusters to repair them. However with change [1] we've introduced
running this task standalone very early during the playbook.
In case cluster is unhelathy - there is no good way to repair it anymore
as
`rabbitmqctl cluster_status` might fail right away with.
So let's ignore failures at this point and run feature_flags only if
we were successfull on fetching the status.
[1] https://review.opendev.org/q/Idf33b6810e1afb3a89dad5e65a48e86ac0a58a67
Change-Id: Ie453f327850f9c9ae54823c17b0292517fa56a82

This reverts commit 29d276582c.
Reason for revert: Eventually the failure, which we were reported previously
was caused not by retries, but by the very early inclusion of the task
from the playbook, which needs to be handled differently. A follow-up
change is needed to address that.
Change-Id: Ib64ff19201944a8581953eed8579abb6c9d9c401

We do have a variable `rabbitmq_erlang_version_spec` which is applicable
only for Debian, while for EL the variable is only private.
This patch does some alignment on things:
* spec is replaced with just a version, as we don't need ternary logic
anymore.
* defenition of erlang version is unified across distros
Change-Id: Ic72212fa2d6df55f2f60d1d38c48310765442e51

Rabbitmq provide a new set of ppa fronted by Cloudflare, use those
instead of the previous one which is no longer mentioned in the
rabbitmq installation guide.
Change-Id: I6eff083b95b392e9f8a05f79ee2ec49705257a64

At the moment we fixup only x86_64 arch while there way more arches
in the wild. So it's worth to have a mapping for architectures in place
rather then maintain quite complex replaces.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/930272
Related-Bug: #2081764
Change-Id: I471fc337c0f13a5b8b10131760ed96032a34f9e8

This is the latest minor version in 3.13 series. Current 3.13.3 does not
exist for noble causing it to install the latest, which is now 4.0
series
Change-Id: Ifaa69e8b9caa8d5dfa9f4b83db31932c82641a40

Instead of using a conditional block, use the "state" parameter
of the yum_repository module to ensure that the repo
config is correctly added/removed if rabbitmq_install_method is
changed after initial deployment.
Change-Id: Ief07a74cd0019fefd58e27fb8dc4299b99299f99

The apt-key module is deprecated so the code is refactored to allow
any of the deb822_repository features to used instead.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-apt_package_pinning/+/927903
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/907886
Change-Id: I96f1f23d6a1cca223ad4fc48a1caa21833a98b22

As task is execured right after handlers execution, it would make sense
to add retries just in case cluster doesn't come up immediatelly back.
Change-Id: Ia26c3a9e9a7a39969a5b6450703c14aad4760eea

As we've backported feature flags enablement to previous releases,
there is expectation that at this point all missing flags were enabled
now.
With that there is no reason to run check each role run, except it is
an expected upgrade of rabbitmq, after which new flags might appear.
Change-Id: Ic004de1ea5d00965e5bebe50b4a93e247b82873d

Change-Id: I4d2bfe34eaddb5e8c8ac6a0ff714ee6da7b7467f

Stable feature flags must be enabled before upgrades to ensure
the service can still start. These additional tasks check for
any disabled feature flags and enable them. This is done
immediately after upgrades so that systems can run with them
enabled striaght away.
Closes-Bug: #2038818
Change-Id: I5211a30899f544a9f7e609e85551a92b245d25bf

Change-Id: Iddc69e81e1677efcd1930afb4dc3113596e9537b

Update the zed release notes configuration to build from
unmaintained/zed.
Change-Id: I220491d548f4fe1031e572de645962238a039207

These jobs aims to ensure that infra can be installed through distro
method
and it still executes it's required functionality.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/917148
Change-Id: I4aa840b93a320bd89f9f630a4f58b8004380f28a

When we do not need to install external GPG keys nor pin packages
to external sources when installation_method is set to distro.
Change-Id: I29bbcacd6ce06516f1ef34cf58da8cac8ef79ea8

This patch adds installation of "native" to EL repo (which actually
is "extras") that provides RabbitMQ and Erlang.
This behaviour would align EL with DEB systems in terms of "distro"
path.
Change-Id: Ifa197131dd010108f3a0745322b32de1fb08b813

Changes in oslo.messaging for 2023.1 exposed a known race
condition in RabbitMQ when dealing with non-HA classic queues.
When a RMQ cluster member is taken down, clients failing over
to other members may erroneously be told a queue exists when it
is in the process of being deleted. This can cause them to
permanently sit waiting for messages from a queue that no longer
exists until their services are restarted.
Making the reply queues HA resolves this issue, at the expense
of a x3 increase in reply queues across the cluster. My
assumption is that reply queues were previously excluded from HA
policy as a performance gain given their link to the number of
compute nodes in an OpenStack deployment.
Context: https://bugs.launchpad.net/oslo.messaging/+bug/2031512
Change-Id: Ia0a26fdfdfa09088c921f1530d4ac020b2bec290

My assumption is that this restart only exists for historical
reasons or in order to force client re-connection and
re-definition of queues to match this policy.
Unfortunately the current task only executes on a single RabbitMQ
node when part of a cluster, so it is ineffective in forcing
reconnection. From a brief review at least when shifting from
non-HA to HA this also isn't required as queues are immediately
updated to reflect the new policy.
Given no clear reason for the task to be present, I'm removing
it ahead of making changes to the default HA policy.
Change-Id: I956abe68d4d7921a4e87ae5a7694d49546bdde94

Policies were always applied to target `all` aspects and
there was no way to have them target only e.g. exchanges.
This can be important though, see [1].
This change enables the use of the apply_to parmeter via the existing
variables while maintaining `all` to be the default.
[1] https://www.rabbitmq.com/docs/parameters#how-policies-work
Change-Id: I61d68c630599b8ef2382663e8a37fdc456435c11

This task is no longer needed in recent releases of openstack-ansible.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/907886
Change-Id: I2190c16548fb44948a26ee2817d8f71fbe66be53

Add the ability to configure the logging options for the rabbitmq-server
via the rabbitmq_log variable and key-value pairs
Change-Id: If9f28f29311150ed7b8ba61d2ddf7e12ca07d3c4

Add ability to set the environment variable RABBITMQ_USE_LONGNAMES
via the rabbitmq-env.conf to be able to use the FQDN for a node
Change-Id: I766bdbe847eb24010bfe9912f07e89d78bbc1bed

This implements a new variable rabbitmq_custom_config to
be able to place custom configuration to the rabbitmq.conf
e.g. to configure installed plugins
Change-Id: I952eefe646b00f60184f8d353f6f055bbdc4ac90

Change-Id: I2e320b862dbc86a70f660510f2b946c6ac135ca1

This implements new variable rabbitmq_erlang_extra_args that enable
deployers to define extra flags for erlang [1]
[1] https://www.rabbitmq.com/runtime.html#cpu-reduce-idle-usage
Change-Id: Ibb0963962d6c7483d54fac69960910c0b13802a5

Change-Id: I4aa03c4ddad206cac8bf73b1ec44eed5aa0f715b

With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.
With that we also update metdata to reflect current state.
Change-Id: I63f6a2c803370736e969aa3a4dea3ea959316def

There was no reason to adjust rabbitmq_package_version previously, as
replaced by the wildcard part is never changed, so patching that doesn't
have any practical sense. We also return `-1` to erlang spec, so that
wildcard would match only the part we expect to change.
This partially reverts I99683a031f935b579d38ae457c484c9a150344c6
Change-Id: Ie2e783d065f32b906ee1554abaf5dc3b24236ca8

For a long time we were struggling with disappearing packages on
packagecloud/cloudsmith repos.
We were told to use {ppa1,yum1}.novemberain.com because packages should
not disappear from there[1].
Unfortunately it just happened causing Rocky jobs to fail with error
message: "No package erlang-25.3.2-1.el9.x86_64 available."
Because we had this issue for a long time and we have not found any
proper solution so far, I think the best we can do is to use wildcards
for version definitions.
Wildcards are used only for build numbers(number after 3th dot) if they
are present. It should minimize a chance to install incompatible erlang
and rabbitmq versions.
[1] https://github.com/rabbitmq/rabbitmq-server/discussions/8386#discussioncomment-6022021
Required-By: https://review.opendev.org/c/openstack/openstack-ansible-os_rally/+/887528
Change-Id: I99683a031f935b579d38ae457c484c9a150344c6

As we target el9 and not el8, we should use the el/9 repo as it is
available, even though it provides RPMs with an `el8` dist tag. This can
help prevent confusion for users who see a repo on the system configured
with `el/8` in the repourl.
There should be no impact on existing installations, as the same repo
content is provided in both repos in almost all cases--namely, a few
early versions of erlang are unavailable, but as we pin to a version
that exists in both repos, there is no impact to this change: at worst,
the erlang package will be upgraded to a newer minor version if a user
somehow has an older one installed.
Change-Id: I70c13f7f882ce8666b12314a31f76fe63599677c

When switching to the Novemberain repos and using the proper EL major
version, some packages are not available in el/9 that are in el/8 repos.
This change bumps rabbitmq to 3.11.17-1 and erlang to 25.3.2-1.
Change-Id: Ia885935ef6d97fae8b61a1c4979b03a51727eb31

Upstream has notified us[1] about the immediate deprecation and removal
of the PackageCloud repositories we are using. Thus, we must move to the
CloudSmith repos already being used for the erlang dependency for
RabbitMQ.
[1] https://github.com/rabbitmq/rabbitmq-server/discussions/8386
Closes-Bug: 2021410
Change-Id: I361757b8f76a6c2d087989a36d8f7543aba69e16

Due to decision made during PTG, we're switching back source for
RabbitMQ to packagcloud, since cloudsmith does rotate versions
too aggressively and we can't keep up with them.
With that we're leaving erlang source to be cloudsmith, since
there're simply no other good sources for Ubuntu/Debian.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879671
Change-Id: Ifac1481b459707b289a9804aa4240ebbf55f6ab7

Using 'always' tag in combination with condition depending on
variables is not always a good idea because these variables may not be
defined at the moment.
Currently, rabbitmq_server role cannot be executed with tag limit,
because task containing 'always' tag has a condition operating on
variable(s) that are registered only when 'rabbitmq-config' tag is being
used.
Change-Id: I7ef7b3fed79933b4c8bbc39a63ca6db54532e10e