Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Broker node EBS provisioned throughput settings #101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
angryhamsterx wants to merge 2 commits into cloudposse:main
base: main
Choose a base branch
Loading
from angryhamsterx:broker_ebs_provisioned_throughput

Conversation

@angryhamsterx
Copy link

@angryhamsterx angryhamsterx commented Dec 24, 2023

what

Ability to enable EBS volume provisioned throughput for broker nodes kafka.m5.4xlarge or larger.
You can add configuration block for this by setting broker_ebs_provisioned_throughput_enabled, and then change enable/disable state by broker_ebs_provisioned_throughput_state.

Caveat:
Once provisioned_throughput block edded, it can not be removed, so there is variable broker_ebs_provisioned_throughput_state to change state of provisioned throughput.

why

  • Support missed configuration options.
  • Additional flexibility in setting up a clusters using this module.

references

Mitrofanov, sbogdanovic, and tommij reacted with thumbs up emoji
@angryhamsterx angryhamsterx force-pushed the broker_ebs_provisioned_throughput branch from ea0fc1a to 385f1f8 Compare December 25, 2023 14:56
ebs_storage_info {
volume_size = var.broker_volume_size

dynamic "provisioned_throughput" {
Copy link

@sbogdanovic sbogdanovic Jan 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case we want to explicitly disable provisioned throughput, to produce for example:

provisioned_throughput {
 enabled = false
}

the module with the change you suggested would try to set volume_throughput to the default value (250).

this would cover that case, and fix this bug:

dynamic "provisioned_throughput" {
 for_each = var.broker_ebs_provisioned_throughput_enabled && var.broker_ebs_provisioned_throughput_state ? [1] : []
 content {
 enabled = true
 volume_throughput = var.broker_ebs_provisioned_throughput_value
 }
}
dynamic "provisioned_throughput" {
 for_each = var.broker_ebs_provisioned_throughput_enabled && !var.broker_ebs_provisioned_throughput_state ? [1] : []
 content {
 enabled = false
 }
}

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@angryhamsterx would you mind if we pull your PR changes into a seperate PR from us, with cloudposse'isms covered, and drive the PR forwards?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hans-d hans-d added wip Work in Progress: Not ready for final review or merge and removed wip Work in Progress: Not ready for final review or merge labels Mar 2, 2024
Copy link

hans-d commented Mar 3, 2024

@angryhamsterx can you update the pr (see also the failing readme check)

@hans-d hans-d added the stale This PR has gone stale label Mar 7, 2024
Copy link

mergify bot commented Mar 9, 2024

Thanks @angryhamsterx for creating this pull request!

A maintainer will review your changes shortly. Please don't be discouraged if it takes a while.

While you wait, make sure to review our contributor guidelines.

Tip

Need help or want to ask for a PR review to be expedited?

Join us on Slack in the #pr-reviews channel.

@mergify mergify bot added triage Needs triage and removed stale This PR has gone stale labels Apr 1, 2024
Copy link
Contributor

@Nuru Nuru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, to fix #117 restore the

 lifecycle {
 ignore_changes = [
 # Ignore changes to ebs_volume_size in favor of autoscaling policy
broker_node_group_info[0].storage_info[0].ebs_storage_info[0].volume_size,
 ]
 }

We do NOT want to ignore changes to provisioned throughput, as that would make it impossible to update.

Comment on lines +34 to +58
variable "broker_ebs_provisioned_throughput_enabled" {
type = bool
default = false
description = "Enable broker node EBS provisioned throughput for broker type kafka.m5.4xlarge or larger"
nullable = false
}

variable "broker_ebs_provisioned_throughput_state" {
type = bool
default = false
description = "Enable or disable provisioned throughput of the EBS volumes for the data drive on kafka broker nodes"
nullable = false
}

variable "broker_ebs_provisioned_throughput_value" {
type = number
default = 250
description = "Provisioned throughput value of the EBS volumes for the data drive on kafka broker nodes in MiB per second"
validation {
condition = var.broker_ebs_provisioned_throughput_value >= 250
error_message = "broker_ebs_provisioned_throughput_value must be minimum 250."
}
nullable = false
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
variable "broker_ebs_provisioned_throughput_enabled" {
type = bool
default = false
description = "Enable broker node EBS provisioned throughput for broker type kafka.m5.4xlarge or larger"
nullable = false
}
variable "broker_ebs_provisioned_throughput_state" {
type = bool
default = false
description = "Enable or disable provisioned throughput of the EBS volumes for the data drive on kafka broker nodes"
nullable = false
}
variable "broker_ebs_provisioned_throughput_value" {
type = number
default = 250
description = "Provisioned throughput value of the EBS volumes for the data drive on kafka broker nodes in MiB per second"
validation {
condition = var.broker_ebs_provisioned_throughput_value >= 250
error_message = "broker_ebs_provisioned_throughput_value must be minimum 250."
}
nullable = false
}
variable "broker_ebs_provisioned_throughput" {
type = object({
enabled = optional(bool, false)
volume_throughput = optional(number)
})
description = "Enable broker node EBS provisioned throughput for broker type kafka.m5.4xlarge or larger"
default = {}
nullable = false
}

Comment on lines +149 to +156
dynamic "provisioned_throughput" {
for_each = var.broker_ebs_provisioned_throughput_enabled ? [1] : []

content {
enabled = var.broker_ebs_provisioned_throughput_state
volume_throughput = var.broker_ebs_provisioned_throughput_value
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dynamic "provisioned_throughput" {
for_each = var.broker_ebs_provisioned_throughput_enabled ? [1] : []
content {
enabled = var.broker_ebs_provisioned_throughput_state
volume_throughput = var.broker_ebs_provisioned_throughput_value
}
}
provisioned_throughput {
enabled = var.broker_ebs_provisioned_throughput.enabled
volume_throughput = var.broker_ebs_provisioned_throughput.volume_throughput
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Nuru Nuru Nuru requested changes

@kevcube kevcube Awaiting requested review from kevcube kevcube is a code owner automatically assigned from cloudposse/contributors

@srhopkins srhopkins Awaiting requested review from srhopkins srhopkins is a code owner automatically assigned from cloudposse/contributors

+2 more reviewers

@tommij tommij tommij left review comments

@sbogdanovic sbogdanovic sbogdanovic left review comments

Reviewers whose approvals may not affect merge requirements

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

triage Needs triage

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /