What is the functional programming method for combining two "streams" of data?

Question

I'm trying to figure out what the appropriate way to deal with this problem is in a functional way. I'm assuming there's a functional data structure or technique that will make this easy to handle.

The problem is that I'm making an API call to get two bits of data, I use one in another API call and if that call works then I use the other bit of data in a second API call. I'm having trouble getting that all to flow together. The API calls can fail of course, so they return Either, and I'd like it to be all one long chain, but I can't figure out how to do that.

In an imperative language I'd do this:

let response = apiCall1();
// error handling, return failure
// do some processing on the response
let json = getJsonBody(response);
let bit1 = json['bit1'];
let bit2 = json['bit2'];
let response2 = apiCall2(bit1);
// error handling, return failure
let response3 = apiCall3(bit2);
// error handling, return failure
return response3 // return success

If the result from each API call was the input to the next function I'd do something like this:

return apiCall1()
 .map(jsonBody)
 .map(apiCall2)
 .map(jsonBody)
 .map(apiCall3)

So what's the functional way to accomplish this?

apiCall1 ----
 | |
 | |
 v |
apiCall2 |
 | |
 | |
 v |
apiCall3 <---|

• Telastyn: I'm not sure the proper name for it, but zip is a function that takes two sequences and returns a sequence of pairs.
• amon: Your "imperative" code seems perfectly reasonable even in a functional code base. If your "imperative" code is not functional, it's because your API calls are side effects, not because you failed to obscure the flow of control with liberal uses of map operations. Using variables for readability is a good thing. This is what Karl's answer is about: syntactic sugar makes it way easier to deal with monads.
• Giorgio: Your implementation is synchronous, not asynchronous, since the main computation only proceeds when it gets a response. To make it asynchronous you'd have to use callbacks or some mechanism like futures / promises. Karl Bielefeldt's answer would still apply, because Futures also provide a flatMap method. If instead of single values you have asynchronous streams of data, you should look into observables.
• CodesInChaos: Do not cross the streams!
• gardenhead: Congratulations, you just discovered monads!

Answer 1

This is where the monadic properties of Eithers come in handy (although you don't have to understand monads to take advantage of them). Most functional programming languages have a way to easily write this sort of chaining. In Haskell it's do notation. In Scala it's for comprehensions. Since I'm more familiar with Scala, I'll demonstrate that below.

val response = for {
 response <- apiCall1().right
 json = getJsonBody(response)
 response2 <- apiCall2(json("bit1")).right
 response3 <- apiCall3(json("bit2")).right
} yield response3

Here, you use the right-projection of each of the Eithers. It basically performs a flatMap on the Right values of the results of the api calls. As soon as it hits a Left however, it will short-circuit the remainder of the calls and yield the Left as the final result.

Note this is just syntactic sugar. The compiler translates it to a series of flatMaps, and you can write it that way if you prefer, but this way is much easier to read.

• Orubel: Karl, technically this is called API Chaining(TM 2013) and requires following the new API pattern of abstracting communication logic/data from business logic to allow for a internal redirect. github.com/orubel/grails-api-toolkit-docs/wiki/API-Chaining
• Karl Bielefeldt: @Orubel, that technique requires server-side support. What I'm talking about is all client-side and is decades older.
• Orubel: Karl, you cannot handle an API chain without a redirect which is handled server side by passing the response to the next endpoint and validating access to the next endpoint; one cannot assume that just because user has access to one endpoint that they have access to ALL ENDPOINTS. So yes, naturally the only way this CAN be done is through the backend. Anything else would be invalid in that you cannot validate and that I/O overhead is the same as making every single request from the client side. The point of an api chain is to reduce those consecutive api calls down to one request/response....
• Orubel: And for the record Karl, monads predate this but they are NOT API Chaining(tm). I have given talks about this since 2012 since I'm the creator and maintainer of the spec.
• Andres F.: Karl, you may be happy to know in Scala 2.12, Either is right-biased (meaning right projections are deprecated) like it should always have been. One step closer to Haskell... bwahahaha!