Best way to add asynchronous execution for already implemented synchronous process

Question

I have a complex process implemented in Java Spring microservice. Currently this process is triggered on user request and it is synchronously executed. This often results in a gateway timeout. Execution can last from 30s to 10min, also I does not want to increase gateway timeout. So I want to execute it asynchronously on a user request and immediately return status 202. There are few options:

• On the controller side spawn a new thread and execute this process in a spawned thread.
• Change the process implementation so it is always executed in a new thread.
• Add a new component like AsyncProcessExecutor which executes process in a new thread.

AsyncProcessExecutor seems like a best option it does not clutter the controller nor already complex process. AsyncProcessExecutor could be extended easly to send slack notification on process completion.

• Greg Burghardt: Is this micro service accepting HTTP calls or does it have a message queue sitting in front of it?
• DimitrijeCiric: No queue, just HTTP API. Developers are users of that microservice. They HTTP API for executing processes. Maybe API call should add a message to a queue which will be used for asynchronous process execution? @GregBurghardt

Answer 1

Execution can last from 30s to 10min

This is too long to execute in an http server. They are designed to send quick responses back to lots of requests.

You need two seperate programs.

1. The http server/REST API, recieve the request, put the message on a database/message queue/file and respond with "OK Working on this! id : 1234"
2. The worker process, poll the database/message queue/directory. When you see a new message, open it up and do your thing. When finished put the result on a database/message queue/file
3. The REST API again, but a different endpoint. Listen for requests asking "is job 1234 done yet?" check the db/mq/result directory for the result of job 1234. Is it there? if yes, open it and send it back in the response.

*obvs im giving the most basic idea here. Sprinkle with your technologies of choice

• Basilevs: I don't get it. HTTP is designed for this. It IS upgradable ro Websockets, it DOES have async status codes and polling headers.
• Ewan: Not really, but post your answer, I'm sure there are more complex and interesting ways of getting around the issues. This is just the super simple approach.
• DimitrijeCiric: Thank you, this seems like a good solution with simple implementation. It could be extended, I could spawn a new process for each request and that could be used to prevent daily restart for already working processes, also it would increase fault tolerance.
• Ewan: if your work is cpu bound then theres not much point spawning more processes, but you can have more computers running the worker and looking at the db/queue for new jobs
• DimitrijeCiric: You are right, but I believe 70% or more is the IO work. We fetch whole tables in batches from 2 databases and compare them.

Answer 2

From the client perspective there are several approaches pull, push and pull and push. Pull is when time to time the client requests a result, an implementation of progress bar principle. Push it is when the server knows its clients and sends the result to each client needing it. Pull and push is when the client first requests a result and after the first client request the server remembers the client and pushes the result to it when the result is available, an implementation of "don't call me I'll call you" principle.

From the server perspective every detail of client perspective flips around, pull turns to push and vice versa.