04 How to Abstract Functions Through Composing Pipeline and Reducer

04 How to Abstract Functions Through Composing Pipeline and Reducer #

Hello, I am Ishikawa.

In the previous lesson, we talked about how to go from abstraction to concretization through partial application and currying. Now, what we are going to talk about today, composition and pipeline, is the process that helps us transform functions from concretization to abstraction. It is a systematic way of encapsulating different component functions into a single function with only one input and output.

In fact, when we were introducing examples of unary functions in the previous lesson, we already saw the embryonic form of composition. In functional programming, the concept of composition is to combine component functions to form a new function.

Let’s start with a simple example of a composition function. For example, if we want to create an isOdd function to determine if a number is odd, we can first write a function that calculates the remainder of the target number divided by 2, and then write a function that checks if the result is equal to 1. In this way, the isOdd function is built on the basis of these two component functions.

var isOdd = compose(equalsToOne, remainderOfTwo);

However, you can see that the order of this composition is counterintuitive because normally, the remainderOfTwo should come first to calculate the remainder, and then the equalsToOne should be executed to check if the result is equal to 1.

So why is there a counterintuitive design here? In today’s lesson, we will answer this question to see how composition and pipeline achieve abstraction, and how the reducer improves the processing performance on a series of operations targeting values.

Compose #

Before talking about composition, let’s first take a look at Point-Free and function components. Here, we will continue to use the “isOdd” function we mentioned earlier, which checks whether a value is odd, and look at its implementation step by step.

Point-Free #

So, what is Point-Free? In functional programming, Point-Free is a programming style, in which “Point” refers to parameters, and “Free” means no parameters. Combining the two, Point-Free means a function with no parameters.

What is the purpose of doing this? In fact, by doing so, we can combine one function with another function to create a new function. For example, to create the “isOdd” function, we can combine these two functions together using this approach and obtain “isOdd”.

var isOdd = (x) => equalsToOne(remainderOfTwo(x));

Function Components #

Next, let’s take a look at function components.

In the code example below, we first define two functions: the first one is “dividedBy”, which calculates the remainder of x divided by y; the second one is “equalsTo”, which is used to check whether the remainder equals 1.

These two functions are actually the component functions we use. You may notice that the characteristics of these two components are that they strive to focus on doing one thing well.

var dividedBy = (y) => {
    return function forX(x) {
        return x % y;
    }
}
var equalsTo = (y) => {
    return function forX(x) {
        return x === y;
    }
}

Then, based on “dividedBy” and “equalsToOne”, we can create two Point-Free functions, “remainderOfTwo” and “equalsToOne”.

var remainderOfTwo = dividedBy(2);
var equalsToOne = equalsTo(1);

Finally, we only need to pass in the parameter x to calculate the corresponding result of “isOdd”.

var isOdd = (x) => equalsToOne(remainderOfTwo(x));

Now, we know that functions can be applied by writing them as component functions. Here, we actually use the declarative thinking of functional programming. “equalsToOne” and “remainderByTwo” not only encapsulate the process, but also remove the parameters, exposing the functionality itself to the user. Therefore, by combining the functionality of these two component functions, we can implement the “isOdd” function.

Independent Composition Function #

Next, let’s take a look at an independent composition function.

In fact, we have already seen the shadow of composition from the previous example. To go further, we can abstract composition into an independent function, as shown below:

function compose(...fns) {
    return fns.reverse().reduce( function reducer(fn1,fn2){
        return function composed(...args){
            return fn2( fn1( ...args ) );
        };
    } );
}

In other words, based on the composition function abstracted here, we can compose the previous component functions together.

var isOdd = compose(equalsToOne, remainderOfTwo);

So, going back to the question mentioned at the beginning of the lesson: Why is composition counterintuitive? Because it is arranged according to the order of passing parameters.

The composition mentioned earlier is actually equalsToOne(remainderOfTwo(x)). In mathematics, composition is written as fog, which means a function takes a parameter x and returns f(g(x)).

Alright, but even if you understand the concept, you may still feel that it is counterintuitive and want a more intuitive order to perform a series of operations. There is a corresponding solution for this, which is to use the pipe in functional programming.

Pipeline #

In functional programming, a pipeline is another way to create functions. The characteristic of a function created in this way is that the output of one function serves as the input to the next function, and they are executed in sequence.

Therefore, a pipeline is processed in a reversed order compared to composition.

Pipeline in Unix/Linux #

In fact, the concept of a pipeline originated from Unix/Linux. Douglas McIlroy, the founder of this concept, mentioned two important points in his article at Bell Labs:

The first point is to make each program focus only on one thing. If there are new tasks, a new program should be built instead of complicating the existing program by adding new features.
The second point is to enable the output of each program to serve as the input to another program.

If you are interested, you can also read this magazine article. Although it is an article from 1978, its design principles are still relevant today.

Now, let’s look at a simple example of a pipeline. In this example, we can find all JavaScript files in the current directory.

$ ls -1 | grep "js$" | wc -l

You can see that this pipeline consists of three parts separated by vertical bars " | “. The first part ls -1 lists and returns all the files in the current directory. The result of this part serves as the input to the second part grep "js$", which filters out all the files ending with “js”. Then, the result of the second part serves as the input to the third part, where we see the final calculation result.

Pipeline in JavaScript #

Now, let’s go back to JavaScript and see how we can achieve the same functionality using a pipeline with the example of isOdd.

Actually, it’s quite simple. We just need to use a reverseArgs function to reverse the order of the input arguments in compose.

You may recall that in the previous lesson on unary, we reduced the number of input arguments of a function to 1. Here, we reverse the order of the arguments and generate a new function. In functional programming, this is considered a classical example of a higher-order function.

function reverseArgs(fn) {
    return function argsReversed(...args){
        return fn( ...args.reverse() );
    };
}

var pipe = reverseArgs( compose );

Then, let’s test whether the pipeline is “smooth”. This time, we arrange remainderOfTwo and equalsToOne in a more intuitive way.

As you can see, isOdd(1) returns true, and isOdd(2) returns false, which is the expected result.

const isOdd = pipe(remainderOfTwo, equalsToOne);

isOdd(1); // Returns true
isOdd(2); // Returns false

Transduction #

After discussing composition and pipeline, there is one more thing I would like to emphasize.

I have repeatedly mentioned that many concepts in functional programming come from the study and control of complex and dynamic systems. Through composition and pipeline, we can further explore the concept of transduction.

Transduction is mainly used in control systems, such as a system where sound waves serve as input, pass through a microphone, are converted into energy by an amplifier, and finally output through a speaker.

Of course, you might not have a clear impression of this term by itself. However, if I mention React.js, you should know that it is a famous front-end framework. The concept of a reducer, which is used in React.js, involves transducing.

In the upcoming lessons, when we talk about reactive programming and the observer pattern, we will delve deeper into the reducer. For now, let’s take a look at the use and principles of transduce and reducer.

So, what is the purpose of a reducer? It is primarily used to address performance issues that may arise when using multiple map, filter, and reduce operations on large arrays.

By using transducers and reducers, we can optimize a series of map, filter, and reduce operations, so that the input array is processed only once and directly produces the output result without the need to create any intermediate arrays.

Perhaps you still find it difficult to understand what I mean. Let’s start with an example that does not use transducers or reducers.

var oldArray = [36, 29, 18, 7, 46, 53];
var newArray = oldArray
  .filter(isEven)
  .map(double)
  .filter(passSixty)
  .map(addFive);
  
console.log (newArray); // Returns: [77,97]

In this example, we perform a series of operations on an array. First, we filter out odd numbers, then multiply them by two, filter out values greater than sixty, and finally add five. During this process, intermediate arrays are continuously generated.

The actual process is illustrated in the left half of the following diagram.

However, if we use a reducer, we only need to perform each operation on each value once to obtain the final result, as shown in the right half of the above diagram.

So how is it implemented? Here, we first take a function, such as isEven, as input and put it into a transducer. The output is a reducer function called isEvenR.

Indeed, this transducer is a classic example of a higher-order function (i.e., a function that takes a function as input and returns a new function)!

In fact, both double and addFive have mapping functionality, so we can use a transducer similar to mapReducer to convert them into a reducer. Similarly, isEven and passSixty have filtering functionality, so we can use a transducer similar to filterReducer to convert them into a reducer.

In abstract terms, the code roughly looks like the following. I will leave the specific implementation as a tease, so you can think about it before our next lesson.

var oldArray = [36, 29, 18, 7, 46, 53];

var newArray = composeReducer(oldArray, [
  filterTR(isEven),
  mapTR(double),
  filterTR(passSixty),
  mapTR(addFive),
]); 

console.log (newArray); // Returns: [77,97]

To sum up, from the example above, we can see that composeReducer serves as a kind of composition function.

Summary #

In this lesson, through an understanding of composition and pipelines, you may have noticed that they are the opposite of some of the applications and currying we discussed in the previous lesson. One moves from the concrete to the abstract, while the other moves from the abstract to the concrete.

However, although their directions are opposite, there is one principle that remains consistent, and that is that each function should have a single responsibility and focus on doing one thing well.

It is worth noting that the difference in direction here does not mean that we should replace the concrete with the abstract, or vice versa. Rather, both are meant to serve the principle of single-responsibility functions and complement each other in the process of concretization or abstraction.

Additionally, through the example of reducers, we have learned how to achieve performance improvement that cannot be achieved through ordinary composition.

In this lesson, we first understood reducers from an abstract level, but you may still feel somewhat unfamiliar with concepts and specific implementations such as map, filter, reduce, etc. Don’t worry, in the next lesson, I will further introduce you to the mechanism of these value-oriented operation tools, as well as functors and monads.

Thought Exercise #

We mentioned that reduce can be used to implement map and filter. Do you know the principles behind this? Feel free to share your answer in the comment section, or if you are not very familiar with this, I hope you can find some information as a preview for the next class.

Of course, you can also discuss your questions in the comment section. Let’s discuss and progress together.