How to design sequential requests to third-party APIs?

Question

I need to implement the following flow:

Back-end makes HTTP request to Service A, waits for a response from Service A,
then makes HTTP request to Service B, waits for a response,
then makes concurrent HTTP requests to Service C, waits for responses.

if the response from Service A does not contain specific info required for the next request, break the flow and return an empty response.
the same is true for the response from Service B.

Services A, B, and C are different external APIs.

Question: Could you suggest a design pattern for this flow? My goal is to make the project maintainable. In case if tomorrow we'll need to replace Service C with Service D, or add new Service E to the flow.

I'm sure failures are inevitable... So my second question is how to handle network errors which may occur during any of the mentioned requests? Should it be a single try-catch block wrapping the flow, or separate try-catch blocks for each request?

I'm using Java/Kotlin with Spring framework if this is important.

---

I'll handle response caching, authentication, token refresh, logging, and configuration in separate classes. This question is specifically about the flow design.

• John Wu: I feel like I'm missing something because this strikes me as a very odd question. If your code needs to make three calls in sequence, you write a line of code that performs the first call, then below it another line that does the next, etc. And if the first response doesn't contain the information needed to proceed, you check it with an if statement and return the empty response. How else would you do it?
• naXa stands with Ukraine: @JohnWu for example, with promises (or CompletableFuture in Java).
• Laiv: naXa, correct me if I'm wrong but, using Spring RestTemplate + HttpClient already makes the solution synchronized. You just make one call after the other. Are you perhaps, asking how to make a pipeline? A data flow?
• naXa stands with Ukraine: @Laiv you're right. Service A and service B use adapters that depend on Spring RestTemplate. Though Service C is different - it uses an adapter that depends on a library (Spotify Web API). Yes, a pipeline or data flow sounds like what I'm looking for.
• Laiv: It might help. Basically, It allows to do what we do in NodeJS. Instead of chaining then one after another, you could resolve the stack recursively.

Answer 1

When talking about a sequence of comparatively expensive tasks, you want to look at the workflow and pipeline patterns. What you will get is a series of independent, individually designed and tested modules that can be combined or composed into a workflow. It will provide component reuse and sometimes is configured externally or dynamically.

You will also gain quite a bit of complexity because someone will provide the threading and the continuation logic But at the end of the day, these services are called in sequence, functionally it will be the same.

My experience is with the .net parallel task library, I think the code example will illustrate the concept well (quoted from MSDN). Here we have two inline functions, one to download, one to process. The runtime by default will run one thread each. It will provide buffering of tasks. The HTTP call is asynchronous, which means the thread is not blocked waiting for the HTTP response.

 var downloadString = new TransformBlock<string, string>(async uri =>
 {
 return await new HttpClient().GetStringAsync(uri);
 });
 // Separates the specified text into an array of words.
 var createWordList = new TransformBlock<string, string[]>(text =>
 {
 // Remove common punctuation by replacing all non-letter characters 
 // with a space character.
 char[] tokens = text.Select(c => char.IsLetter(c) ? c : ' ').ToArray();
 text = new string(tokens);
 // Separate the text into an array of words.
 return text.Split(new[] { ' ' }, StringSplitOptions.RemoveEmptyEntries);
 });
 downloadString.LinkTo(createWordList, linkOptions);

I found a thing called activiti that might be useful? Again, this might be overkill, depending on how often your workflow actually changes, and you could just work with promises for a cleaner implementation.

Answer 2

You can wrap each Service into it's own class, where all the classes implement a common interface. This would allow you to potentially change the service in the future without the client code (i.e the code that organises the flow) knowing which service is being called. The client code would only need to know that these 3 services need to be called in a particular sequence.

Network errors are potentially already handled by Java and would simply result in the request failing altogether.