Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Feature request: Support raise_on_entire_batch_failure in SqsFifoPartialProcessor #7596

Open
Labels
batchBatch processing utility feature-requestfeature request
@kellen-schmidt-self

Description

Use case

BatchProcessor supports a raise_on_entire_batch_failure option (through its parent class, BasePartialBatchProcessor). SqsFifoPartialProcessor extends from BatchProcessor, but doesn't allow the raise_on_entire_batch_failure option to be set, so the default value of True is always used.

For the same reason that raise_on_entire_batch_failure is a useful option on BatchProcessor (as explained in the docs), that option would be equally useful on SqsFifoPartialProcessor. We want to configure both Standard and FIFO queues to behave the same (not raise BatchProcessingError when all records in the batch fail to process), but right now aws-lambda-powertools only supports that option for Standard queues.

Solution/User Experience

A simple solution would be to add raise_on_entire_batch_failure to the constructor of SqsFifoPartialProcessor and pass it when the superclass constructor is invoked:

class SqsFifoPartialProcessor(BatchProcessor):
 def __init__(self, model: BatchSqsTypeModel | None = None, skip_group_on_error: bool = False, raise_on_entire_batch_failure: bool = True):
 """
 Initialize the SqsFifoProcessor.

 Parameters
 ----------
 model: BatchSqsTypeModel | None
 An optional model for batch processing.
 skip_group_on_error: bool
 Determines whether to exclusively skip messages from the MessageGroupID that encountered processing failures
 Default is False.
 raise_on_entire_batch_failure: bool
 Raise an exception when the entire batch has failed processing.
 When set to False, partial failures are reported in the response

 """
 self._skip_group_on_error: bool = skip_group_on_error
 self._current_group_id = None
 self._failed_group_ids: set[str] = set()
 super().__init__(EventType.SQS, model, raise_on_entire_batch_failure)

Alternative solutions

Acknowledgment

Metadata

Metadata

Assignees

No one assigned

    Labels

    batchBatch processing utility feature-requestfeature request

    Type

    No type

    Projects

    Status

    Backlog

    Milestone

    No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

        AltStyle によって変換されたページ (->オリジナル) /