The commit message? 5 simple words. Build electron apps: fix builder

Let me back up.

Recently, a vulnerability was discovered in BitPay’s Bitcoin wallet, Copay. You can see coverage at ZDNet here.

An NPM module called flatmap-stream was found to be the culprit. Someone pushed an update that contained JavaScript malware designed to steal funds from a user’s Copay wallet.

The malware itself is fascinating, delivering an encrypted payload that would only activate when included as a library in a specific project. But I’ll leave that for another post, for now I’m more interested in how this malicious payload found itself nestled within the Copay codebase.

Back to the commit.

This was no small change. A large change, in fact. With 1,661 additions and 2,482 deletions, this 3-file change pulled in new versions of a great deal of third-party libraries. It just so happened that one of those contained malware that pilfered bitcoins out of BitPay user wallets. The package in question: flatmap-stream.

Before and after, can you spot the exploit?

Interestingly enough, the exploit was contained only in the minified version of flatmap-stream. That Copay’s build process relies on the minified source code of transitive dependencies is concerning, to say the last.

A chain of automatic updates

Now, in order to understand flatmap-stream’s role in all this, we must first look at event-stream. event-stream is an extremely popular package, with almost 2 million weekly downloads from NPM. Despite its popularity, the creator had moved onto other projects and left event-stream to wither on the vine. Not at all uncommon in the open-source, new development is always more fun than triaging bugs and giving tech support.

A user by the name of right9ctrl offered to take over maintenance of event-stream, and the current owner, dominictarr obliged. right9ctrl made a few improvements, and notably added a dependency for flatmap-stream, a project he controlled. The dependency used was ^0.1.0, that will be important later.

This is where things get interesting. Some time later, a modified version of flatmap-stream was pushed to NPM, containing a chunk of malicious encrypted JavaScript. It was published as version 0.1.1, but there were no commits to the flatmap-stream git repository. Very suspicious.

Because event-stream used a carat dependency, it picked up the new 0.1.1 version with the bugged code. That is, on fresh installs of event-stream the bugged version of flatmap-stream would be used.

Similarly ps-tree depended on event-stream with ~3.3.0, so it picked up the new bugged version of event-stream.

npm-run-all depended on ps-tree with ^1.1.0, so it picked up the new bugged version of ps-tree.

Finally Copay, BitPay’s Bitcoin wallet, had a dependency on npm-run-all.

Here’s that dependency chain again:

Bugged version of flatmap-stream published => event-stream => ps-tree => npm-run-all => copay

At this point I want to note that nothing said thus far implicates right9ctrl. It could have been that flatmap-stream’s NPM credentials were cracked, and this was just a case of tilde- and carat-dependencies gone horribly awry. I think people are jumping the gun on this one and will reserve judgment.

3rd party dependencies

“I have always depended on the kindness of strangers” – BitPay

Now back to the copay commit. What it shows is that BitPay has little to no auditing of third-party dependencies. How am I so sure? Because that package-lock.json update shows a version bump for flatmap-stream from 1.1.0 to 1.1.1. Any attempt to track down the source code for version 1.1.1 for even the most cursory inspection would have shown an artifact with no link back to version control, a huge red flag.

I realize that development teams have limited resources. I also realize that copay has 2729 external dependencies at last count. But this exploit had a real tangible cost, and as developers we should be asking ourselves how we can prevent incidents like this from occurring.

Action items

1. Use package-lock.json

If you’re not using package-lock.json, or your programming language’s analog, use it. Here’s more information on what it’s all about. Even if you don’t audit all your dependencies, at least you can snapshot them so they’re consistently applied.

2. Beware of minification

You can’t audit minified code. So you had better be minifying yourself, rather than relying on it from your third-party providers.

3. Justify your dependencies

There’s a lot of gradient between “re-write ExpressJS” and “re-write left-pad.” Evaluate your dependencies on the basis of popularity, time since last commit, complexity, and reimplementation time estimate. I’m not saying always re-invent the wheel, but just be aware of the trade-offs.

4. Break up your commits

Conclusion

This incident is sure to provoke discussions around long-term maintenance of open-source projects, and responsibly importing dependencies. One of the reasons why NodeJS is such a joy to work with is because there are free libraries out there for pretty much anything you can imagine, but NPM administrators simply don’t have the resources to properly vet every package they serve. Hoping for the best might work out for a toy hobby project, but an open-source Bitcoin wallet is a tremendously high value target. Expect to see more abandoned projects weaponized in the coming months.