Seeking refuge from unsafe JavaScript

In late 2013, the vast majority of Plaid's code was JavaScript. Though the proportion has decreased since then, JavaScript still accounts for more than half our code.

Early Plaid code was fairly typical imperative JavaScript. For example:

1module.exports = function(raw)
2{
3  var o = {};
4
5  for (var i = 0; i < raw.values.length; i++) {
6    if (raw.values[i].id === 'date')
7      o.date = Date.create(raw.values[i].value);
8    else if (raw.values[i].id === 'number')
9      o.number = parseFloat(raw.values[i].value)*-1
10  };
11
12  return o;
13}
14

Soon after I joined we began using Underscore in our various projects.

We learned to always provide {} as the first argument to _.extend to

avoid unintentional mutation. We accepted the fact that _.chain worked

with Underscore functions but not functions defined elsewhere. We tolerated

Underscore's inconvenient argument order, and even added placeholdersupport to _.partial to make point-free programming with Underscore

possible (though not natural). Using Underscore was clearly better than

using no library at all. We were contented.

Underscore saved us from writing for loops, but did nothing to fix our type

errors.

And we had many type errors.

We were able to avoid some type errors by using a linter and incorporating

linting into our pull request workflow, but a significant proportion of type

errors cannot be detected statically in a language as dynamic as JavaScript.

Improving our test coverage further mitigated regressions.

Even after mitigating type errors caused by human error, many type errors

still resulted from inconsistent data. At Plaid we process data from many

disparate sources, but even for a single source the data's shape may vary.

We were bitten by this several times. Take the following expression:

1data.foo.bar.baz
2

This looked innocuous, and the test suite passed. We deployed, and then noticed

this in the production error logs:

1TypeError: Cannot read property 'baz' of undefined
2

Ah. So data.foo.bar could be undefined in some cases, apparently. So:

1data.foo.bar && data.foo.bar.baz
2

Fixed? We then learned that data.foo was undefined in some cases.

We resorted to using guards:

1data && data.foo && data.foo.bar && data.foo.bar.baz
2

As Underscore became an ever more important ingredient in writing JavaScript

programs at Plaid, we grew tired of function expressions cluttering our code

(arrow functions were not available to us at the time). To define a function

that sums a list of numbers, one might write the following in Haskell:

1foldl (+) 0
2

With Underscore, one might have written:

1_.partial(_.reduce, _, function(a, b) { return a + b; }, 0)
2

As a side project I created Nucleotides, a tiny library which makes every

JavaScript operator available as a function. We could then write:

1_.partial(_.reduce, _, nucleotides.operator.binary['+'], 0)
2

This was still much less clear than the Haskell equivalent.

One day Graeme Yeates mentioned me on a Ramda issue. It was my first

exposure to Ramda, and I was impressed by its terseness:

1R.reduce(R.add, 0)
2

Elegant! Like the Haskell definition, it takes advantage of currying.

Several of us at Plaid caught the Ramda bug, and Plaid became one of the

first companies to use Ramda in production. No longer did we need to worry

about functions mutating their arguments, as Ramda functions never did.

No longer was _.partial necessary to define specialized functions

in terms of more general ones, as every Ramda function has support for

partial application baked in.

Once again, we were contented.

I then began to worry about Ramda's unsafe functions. R.head,

for example, is of type [a] → a. When applied to [], there is no a,

so Ramda returns undefined. This means evaluating an expression such as

R.toUpper(R.head(xs)) will result in a run-time exception if xs is [].

This problem can be resolved by changing the type of head to [a] → Maybe a.

If head is applied to [] the result is Nothing(), which indicates a

failed operation. If head is applied to ['x', 'y', 'z'] the result is

Just('x'). The head of the list, 'x', is wrapped in a container which

indicates a successful operation.

Nothing() and Just('x') are both members of the Maybe String type. Both

values support exactly the same set of operations. To transform the String

which may be inside the Maybe, we use map:

1                   +------+                 +--------------+
2['x', 'y', 'z'] ~~~| head |~~> Just('x') ~~~| map(toUpper) |~~> Just('X')
3                   +------+                 +--------------+
4
5                   +------+                 +--------------+
6             [] ~~~| head |~~> Nothing() ~~~| map(toUpper) |~~> Nothing()
7                   +------+                 +--------------+
8

To quench my thirst for type safety I defined safe versions of several unsafe

Ramda functions (including head). Initially these lived in a file of helper

functions in one Plaid project. We realized these would be useful in other

projects and to people outside the company, so we released Sanctuary on

GitHub and npm. Now, with Ramda and Sanctuary, it's possible to write terse,

declarative programs that work correctly for all inputs.

1S.pipe([S.gets(String, ['transaction_info', 'amount']),
2        R.chain(N.normalizeAmount),
3        R.map(S.negate),
4        S.or,
5        R.over(L.amount)])
6

This function, of type Object → Tx → Tx, describes a sequence of

transformations to safely extract a particular value from an Object,

and possibly update the value of the amount field of a Tx value.

It acknowledges the following possibilities:

• the transaction_info field may be absent;

• the value of the transaction_info field may be null or undefined;

• the amount field may be absent;

• the value of the amount field may not be of type String(the argument type required by N.normalizeAmount); and

• the value of the amount field may not actually represent an amount.

It does so with the Maybe data type rather than with

incoherent guards and exception handling.

Although Sanctuary was initially developed internally, several people from

outside Plaid have become valued collaborators since we released the project

under the MIT license. Stefano Vozza was the first external contributor

(documenting much of what was a completely undocumented API at the time), and

remains one of the most active. Kevin Wallace contributed the wonderful

multi-line error messages:

1S.fromMaybe(0, S.Just('XXX'));
2// ! TypeError: Type-variable constraint violation
3//
4//   fromMaybe :: a -> Maybe a -> a
5//                ^          ^
6//                1          2
7//
8//   1)  0 :: Number, FiniteNumber, Integer, ValidNumber
9//
10//   2)  "XXX" :: String
11//
12//   Since there is no type of which all the above values are members, the type-variable constraint has been violated.
13

In recognition of the fact that the Sanctuary community is now self-sustaining,

we've transferred the repositories to the sanctuary-js organization on

GitHub.

I believe these projects have an important role to play in the future of

functional programming in JavaScript, and in exposing JavaScript programmers

to ideas from other languages. I'll continue to work alongside other members

of the community to improve our refuge from unsafe JavaScript.