How to "pass through" data in a functional programming pipeline so that it's accessible later on in the pipeline

Question

I am trying to refactor some JavaScript code to use functional programming principles.

I have some functions that I want to use in a series of maps.

const transformedData = rawData
 .map(getIdFromRawData)
 .map(getCachedDataById)
 .map(transformData)

getIdFromRawData takes in rawData:

{
 id: 123,
 name: "taco",
 is_tasty: true
}

And returns 123.

getCachedDataById takes in an id like 123 and returns cachedData (it's an impure function that gets the data with an I/O operation):

{
 transformedValue1: "cheese",
 transformedValue2: "taco mix",
 transformedValue3: "salsa",
 ...
}

The final function in the pipeline, transformData, needs to take in both cachedData and rawData:

function transformData({ cachedData, rawData }) {
 if (Object.keys(cachedData).length) {
 // use cachedData to skip some work while transforming
 } else {
 // use rawData for transformation
 }
}

But I didn't pass along rawData in the return value of getIdFromRawData. I didn't do that for the following reasons:

• getIdFromRawData is used elsewhere and needs to have a generic signature.
• easier to test getIdFromRawData
• function name getIdFromRawData does what it says on the tin

I want to have some kind of "pass through" functionality with my pipeline. In researching how best to handle this, I've seen mention of using a reader monad, but I'm not sure how to implement it for this use case, or if it's even the right approach.

Something like this initially comes to mind:

function passThrough({ data, func, key }) {
 const result = func(data[key])
 return { ...data, ...result }
}
...
const transformedData = rawData
 .map((data) => passThrough({ data, func: getIdFromRawData, key: 'id' }))
 .map(getCachedDataById)
 .map(transformData)

This doesn't quite do what I want, though, because now getCachedDataById is receiving an argument shaped like an object ({ ... }) and not a number (123).

rawData needs to be present and accessible throughout the pipeline, but it only is relevant at certain parts of the pipeline.

Is there a certain kind of monad that applies to this?

Are monads not really relevant here?

Is there another approach for this?

• user1937198: Your function passthrough is touching on the category theory concept of arrows (haskell.org/arrows) specifically the first function.
• rpivovar: This is really interesting -- thanks. I'm still unclear on what exactly a monad is, and how that differs from this arrow. I'm really debating learning a new language to bolster my understanding of functional programming. Maybe it should be Haskell?!

Answer 1

You can use Lambdas and Anonymous types.

const transformedData = rawData
 .map(i => (id: getIdFromRawData(i), raw: i))
 .map(i => (cachedData: getCachedDataById(i.id), raw: i.raw))
 .map(i => transformData(i.cachedData, i.raw))

Now I'm not constrained by the parameters and return type of my functions as the lambda function and anon type allow me to wrap the underlying function with another function without having to define it.

*I dont always use i as an iteration variable, but when i dont it's a j/k

• rpivovar: This seems so obvious and yet I couldn't figure out on my own. Thank you

Answer 2

Instead of just chaining map, would it make sense to do something like this?

const ids = rawData.map(getIdFromRawData);
const cachedData = ids.map(getCachedDataById);
const transformedData = ids.zip(cachedData).map(transformData);

Answer 3

Usually in functional languages you would use something like do notation in this situation. However, remember you can always combine maps, like (forgive my syntax):

const transformedData = rawData.map(raw => {
 const id = getIdFromRawData(raw);
 const cached = getCachedDataById(id);
 transformData(raw, cached);
})