You can finally "npm link" packages that contain peer dependencies!

For the past year I've been dealing with an annoying bug in node/npm that there has been no simple solution for. It has to do with npm link and peer dependencies. Before I talk about the issue let me briefly explain what both of those things are for the uninitiated.

npm link

Npm contains a command called link that is extremely convenient for developing modules you intend to use in other projects and testing them within those projects. Say you have a project called Foo and Foo depends on a module you're building called Bar. Typically to test any work you've done on Bar you'd need to install it in Foo from the npm registry or manually copy it into Foo's node_modules directory. With npm link it becomes possible to symlink Bar into Foo's dependencies so that you can develop on Bar without having to continually copy changes to the Bar you copied into Foo's node_modules. To do this you simply navigate to Bar's directory on your machine and run npm link by itself. This will create a symlink in your global node_modules directory that points to the actual location of Bar on your machine. This is also useful for command line tools you wish to test out because once they are symlinked into the global node_modules directory you should be able to use the command like any other command installed globally from npm.

Global CLI modules aren't the only things this is useful for though. You can then navigate to Foo's directory and run npm link Bar. This will create a symlink in Foo's node_modules directory that points to the global symlink created for Bar. After that you can happily develop on Bar where you normally do your work and Foo will see the changes right away without you having to copy anything.

Peer Dependencies

You probably know about dependencies and dev dependencies in package.json, but you may not have known about peer dependencies. Peer dependencies can be added to your package's package.json file by simply adding a peerDependencies: { } node to it and listing dependencies like you normally would. The difference here is that npm will not try to install these. Instead you are telling node that this module expects to be installed side by side alongside another module in a parent project. To demonstrate let's go back to our Foo example which depends on a module called Bar. Now let's say that Foo also depends on another module you're developing called Baz. Finally, let's say Bar also depends on Baz. In the Foo project we require Baz and setup a bunch of configuration for it. Bar also depends on Baz but we don't want to have to setup configuration for Baz all over again when we require it from Bar so we need the module to be the same instance of the module that was required by Foo. If we listed Baz as a normal dependency to Bar then Bar would end up installing a second copy of Baz in its own node_modules directory.

Utilizing peer dependencies helps us solve this situation. By listing Baz as a peer dependency in Bar's package.json we are telling node that Bar expects to have Baz installed alongside it in whatever parent project is requiring it. Foo in our example case. This peer dependency functionality is very useful for plugin system architectures. For instance Gulp is a module for creating build tasks. There are lots of plugins for Gulp that have been published to the npm registry. Most Gulp plugin modules list Gulp as a peer dependency in their package.json, informing the user of the plugin that it expects to have Gulp also installed in your project if you are installing the plugin.

The Problem

Now that you know about npm link and peer dependencies, what do you suppose happens project Foo tries to npm link Bar who has a peer dependency on Baz? The link seems to work fine until you try to run Foo. Foo will successfully require both Bar and Baz dependencies like normal, but when it gets to the line in Bar where it tries to require Baz your program will crash and tell you that it was unable to find module Baz. This is because Bar was installed into Foo using npm link so that only a symlink exists for Bar in Foo's node_modules. When node tries to walk the directory tree and find Baz it will fail to find it because it's only walking the directory tree for Bar wherever it is actually located on your machine. Node can't find the peer module because it doesn't keep track of the fact that it required Bar via a symlink and that any requires from Bar should travel back to the directory on the other end of the symlink.

This has caused us issues at work because we are in this exact situation. Up until now we've had to do it the classical way and simply cut/paste Bar into Foo's dependencies. Since we actually copied the files there it has no problem walking the directory tree to require peer dependencies. It works fine doing it this way but it's a huge pain sometimes. I would often find myself tweaking the code I copied into Foo's node_modules only to forget I was making changes in there and accidentally run npm install inside Foo. This blows away the modifications I made to the module in question and overwrites it with the module from the registry. If only npm link worked with peer dependencies properly.

The Fix

A fix was proposed in the node repository and a PR was submitted. It was about to make it into node v6 when suddenly a comment battle on the pull request broke out because apparently fixing this quirk of node broke a bunch of 3rd party module loaders taking advantage of some edge case caused by this bug. If you ask me I would say it should up to the 3rd party module loaders to adapt to these changes, but I digress. Ultimately they decided to revert the fix. About a month went by before this commit finally landed in node to address this issue by the use of a CLI flag. It's not the seamless fix I was hoping for but it works so I'm happy.

We can finally npm link modules that have peer dependencies. The only caveat is that you need to run your project using the --preserve-symlinks flag. So instead of running node app.js you'll want to run node --preserve-symlinks app.js. With that included we no longer see errors where Bar is unable to require Baz when Bar was required by Foo via a symlink :D

I'm a little put off by the fact that I need a flag to fix what I would consider to be an obvious bug, but it's a small price to pay for no longer having to manually copy code back and forth and keep two directories of the same code in sync with one-another. If you too have encountered this bizarre edge-case in your development then I hope this helps provide you some relief :)