I'm writing a microservice (in .netcore) that will handle some requests using WebAPI, but also it needs to react to some external events. I see 2 possibilities how to solve it (on a very high level):
Have a lambda that is triggered by SQS message, and put event(s) handling logic in that lambda function.
Have a background process in my WebApp project (the same that hosts webAPI) which will be listening (reading) from SQS.
Are there any strong arguments to favour one approach over another? Or is it strongly opinionated?
1 Answer 1
Lambda has numerous advantages over a background worker:
Lambda will only run when there's something in the queue & die immediately after message processing completes. A background worker is always running.
Letting SQS message events trigger Lambda is a much better approach than a background worker continuously polling the queue, unnecessarily consuming CPU cycles.
Even if the Lambda runs thousands of times, it'll cost negligible compared to what it'll cost you to keep the background worker running all the time.
Lambda will scale seamlessly in proportion to the number of messages in the queue. The background worker's message processing capacity is limited by the resources (CPU/memory) of the system it's running on.
Lambda will finish processing faster compared to a background worker if the queue contains a large number of messages. Several instances of the Lambda will be triggered in parallel to process the large number of messages but the background worker won't process batches of SQS messages in parallel, like Lambda does.
Lambda will never interfere with your web API. The background worker will compete with the web API for system resources. In extreme cases, the web API might go down if the background worker is processing a large number of queue messages.