Higher-order functions in Lodash

In this article I would like to explain concept of higher-order functions and how they are omni-present in my favorite Javascript library: Lodash.

Higher-order functions are a great way of making code more flexible and reusable, as well as making it more declarative. Before we jump to the definition, let’s take a look at a simple example. We will define function that multiplies two values:

function multiply(a, b) {
  return a * b;
}

Let’s play a little bit with this function:

> multiply(21, 2)
< 42

Suppose, that we often use this function to double a given value. We can create then new helper function like this:

function double(v) {
  return multiply(v, 2);
}

Now we can easily double values:

> double(5)
< 10

What we see here is a classical function delegation (one function delegates to another).

Yet, it is not the only possible way to accomplish similar result. In languages in which functions are first class citizens (such as Javascript), there is another, more functional way to define double() function. It is known as partial function application.

Partial application

According to Wikipedia:

partial function application [...] refers to the process of fixing a number of arguments to a function, producing another function of smaller arity.

With partial function application we can create double() function in the following way:

var double = partial(multiply, 2);

partial() takes a function passed as the first parameter, binds some of its parameters to a fixed value and returns new function with reduced arity (number of parameters). Let’s speculate how implementation of partial in JavaScript could look like.

Our function takes and stores in local variable (fixed) all parameters except the first one (1). Then it returns a new function (2) that calls our original function fn with a parameter list prepended with parameters stored in the fixed variable (3). This vanilla JS implementation is simplistic yet powerful.

But wait — notice ugly invocations of slice.apply(). They are necessary because arguments object in JavaScript is not a real array, therefore it doesn't have slice method, so we use Function.prototype.apply().

But if we used ECMAScript 2015 (ES6), we could simplify code by using the rest operator:

However, if we want to stick to ES 5, we can rewrite this function and already benefit from using Lodash. This is because Lodash accepts arguments as a function parameter wherever array parameter is expected. You can also easily convert any array-like object to actual array using _.toArray(). Our enhanced implementation would look like this:

But luckily, we don’t have to write our own implementation of partial, as Lodash already has one: _.partial(). Moreover, as you will see in next example, it is much more powerful than this simplistic one.

Say we have a divide() function that divides two numbers, defined as:

function divide(a, b) {
  return a / b;
}

Now we would like to reuse our divide(a, b) function to create a new half(n) function that divides given number in half, just as we did in our previous scenario. At a first glance it is similar to our multiple/double example. However, code:

var half = _.partial(divide, 2);
> half(4);
< 0.5

… returns, as you can see, invalid results.

This happens, because earlier order of parameters was irrelevant (multiplication is commutative). Now we need to fix the second parameter and leave the first parameter loose. We cannot do it with our naive implementation, but fortunately the authors of Lodash took it into consideration and provided a solution for such cases. In Lodash we can skip parameter binding using a placeholder like this:

This way we created two new functions: one that halves and another one that inverts:

> half(5);
< 2.5
> invert(5);
< 0.2

Curring

Lodash offers one more function similar to _.partial(). This function is called _.curry(). Let's give it a try:

> var divideC = _.curry(divide);
> divideC(4, 2)
< 2
> divideC(4)(2)
< 2
> divideC(4)
< [Function]
> var half = divideC(_, 2);
> half(4)
< 2

After transforming a function with _.curry() we get a brand new function that will accumulate and fix parameters from subsequent invocations until all expected parameters are specified, in which case original function is invoked. Parameters can be specified one-by-one or in batches. As you can see, you can also skip parameters using _ placeholder, just as with _.partial().

Limitations of _.curry()

_.curry() is more powerful than _.partial() but it also has some limitations. Take a look at the following example:

What happened here? Well, parseInt(string, radix=10) has a second, optional argument. Lodash cannot tell what is actual arity of function, and will assume arity based on Function.prototype.length property which is equal to number of parameters specified in function declaration. Similar case applies to situation when function accepts variable number of parameters using arguments object (a.k.a variadic parameters). This can lead to very unexpected and error-prone behaviors. In such cases it is recommended to provide correct arity of function when curring:

Other higher-order functions in Lodash

_.partial() and _.curry() are great example of higher-order functions as both of them accept and return functions.

Higher-order functions are functions that accept (as parameters) and/or return other functions.

The entire Lodash library is full of higher-order functions. The most notable of them are: _.identity(), _.negate(), _.memoize(), _.constant(), _.property(), _.iteratee(), _.matches(), _.conforms(), _.overSome(), _.overEvery(), _.flow().

If you use Lodash (or plan to use) on a daily basis it is worth to be aware of their existence, as they considerably reduce the amount of code that needs to be written. Furthermore, they contribute to code readability improvement. I won’t go into details of describing each of those functions as Lodash docs are excellent in this matter. Let me make only one exception: _.flow(). Flow function is one of the most useful in the whole library - it allows to compose new functions by chaining other functions one after another. Output (return value) of every subsequent function becomes an input for the next function in the sequence. It is similar to pipe (|) operator in Linux bash. From mathematical point of view it is a classical function composition:

Thanks to _.flow() it is easy to assemble new functions from existing ones:

_.flow() is similar to _.chain(), however unlike _.chain(), which binds to data in the very first invocation, the result of _.flow() is a function that accepts data at the end. This implies it can be assigned to a variable or passed as a parameter, allowing it to be effectively reused for different data sets.

Below you can see code that enumerates 5 countries with largest cities in the world. I have used the ES 2015 arrow function notation for brevity:

cities.json contains data about 91 largest cities in the world. Data about population was taken from Wikipedia.

Now, let's use _.partial() and _.curry() to rewrite this example:

Alternatively, we could define as var greatherThan = _.curryRight(_.gte). _.curryRight() is similar to _.curry() but it binds parameters in reverse order (starting from the last one).

Furthermore, for functions which accept iteratee argument (like _.map(), _.countBy(), _.groupBy()), Lodash automatically wraps the iteratee argument with an _.iteratee() function, which for string parameters eventually delegates to _.property() function. So our code can be shortened even further:

Lodash/fp

Once I became comfortable with _.curry() and _.partial(), I've noticed that most of the time I curry most of the functions. Moreover, I tend to skip the first parameter (or use *Right() variant of aforementioned functions).

Then I stumbled upon lodash/fp variant of Lodash, which

promotes a more functional programming (FP) friendly style by exporting an instance of lodash with its methods wrapped to produce immutable auto-curried iteratee-first data-last methods.

Lodash/fp basically introduces the following changes:

1. curried functions - all functions are curried by default,
2. fixed arity - all functions have fixed arity, so they don't cause problems(as shown before) with curring. Any functions that have optional parameters are split to 2 separate functions (e.g. _.curry(fn, arity?) is split into _.curry(fn) and _.curryN(fn, arity)),
3. rearranged parameters - functions parameters are re-arranged so data is expected as the last parameter, because in real life use cases most of the time you want to fix iteratee parameters and leave data parameters open,
4. immutable parameters - functions no longer mutate passed parameters, but return altered copies of objects,
5. capped iteratee callbacks - have arity caped to 1, so they don't pose problems with curring (see 2.),
6. no more chaining - function chaining using _.chain() or _() is no longer supported. Instead _.flow() can be used.

For more details and motivation for each individual change go to Lodash FP guide.In a nutshell all of those changes together result in much more declarative, error-prone and boilerplate-free code.

Once again our example rewritten to lodash/fp style would look like this:

As you can see, there is no more _.curry() invocations as functions are curried by default. _.chain() invocation has been replaced with _.flow() and cities parameter is passed at the end. If we would saved result of _.flow() to a variable, then we could reuse it for different cities dataset.

Summary

Understanding higher-order functions, especially _.partial() and _.curry(), is crucial if we want to benefit to the maximum extent from functional libraries like Lodash.

In my upcoming article I will show that Lodash is not only a list manipulation library.

I will show how higher-order functions and customizers can improve readability at the same time reducing number of lines of code.

Find Pragmatists on Facebook and Twitter