Notesnook was not always a monorepo with 10s of subprojects. Everything used to be in its own Git repository. While that often became tiresome, it was also simpler in a way.
In October 2022 when we went open source, our beast of choice for the seemingly complicated task of monorepo management was Lerna (and Nx). We had to make a few concessions, learn some new patterns, give up old habits, but it all worked out...eventually.
What we had not foreseen was the impact it would have on our productivity. The only reason we decided to use Lerna was for its package management in a monorepo structure. At the time it solved the problem of, "How do I install X subproject in Y?" but every day we had to face some new problem requiring us to either work around Lerna or deal with it by changing something else. A few examples:
- It is not a trivial task to use Lerna with
- Sometimes Lerna would refuse to update the package lockfiles. The solution? Delete all the
node_modulesdirectories and run
- CI would fail because Lerna automatically switched to
npm ciwhich requires the lockfile &
package.jsonto correlate. The solution? Force a lockfile update, either by clearing all the
node_modulesor deleting the lockfiles.
gitbranches and bootstrapping again was a nightmare. Sometimes dependencies wouldn't install or Lerna would install old versions due to outdated package lockfiles. The solution? Delete
node_modulesdirectory and run
- There's no way to do
npm iin a subproject when you are using Lerna. You have to use
lerna adddoesn't work well with
file:packages requiring you to unnecessarily publish everything to NPM.
- There's no
lerna removeso the only way to uninstall a package is by removing it from the
package.json, but most of the time that also requires deleting the whole
- Lerna takes 100 MB after install.
Lerna has forced us to download gigabytes over and over again. Okay, I admit, it's not that bad - there's caching and all but still I shouldn't have to clear out
node_modules repeatedly just to get package management to work.
I am aware that Lerna is not only meant to be used for package management. In fact, even Lerna had to acknowledge their broken package management, calling it "Legacy", and encourage users to use the package managers' Workspaces feature instead. From v7 onwards, Lerna no longer provides its users with the ability to bootstrap or install a package through it.
It's good advice when you are starting a new project, but in an old project that's "working" there's no time to deal with all the issues that come with using Workspaces:
npmworkspaces defaults to hoisting all the packages in a workspace to the root
node_modulesdirectory. I have found no way to turn it off. Hoisting creates a boatload of issues which require more tools to fix, which create a new set of issues, ad infinitum.
yarnworkspaces supports turning off hoisting and was almost perfect, but it failed to install some native Node.js dependencies. I had no time to look into why it broke.
pnpmalmost worked, but it isn't supported by
flatpak-builderand also requires some nasty hacks to work with Metro Bundler — the de facto bundler for React Native projects.
bun installdoesn't work on Windows.
We needed a tried and tested solution that "just worked". Our requirements:
- It should be possible to do
pnpm installwherever we want.
- Bootstrapping should be quick and responsive (i.e., we shouldn't have to see dead progress bars for ages).
- No learning curve. Everything should be obvious, boring, and simple.
- No hoisting. All packages should be placed in each project's
node_modulesdirectory. This is widely supported and very stable.
- Small & hackable, requiring minimum maintenance.
- Easy to bootstrap specific scopes (e.g. only bootstrap the desktop app).
- Reliable cross-platform support.
- Zero config files.
- It shouldn't be 100mb in size.
You must be thinking (with a groan), "Here comes another monorepo management tool". You'd be 100% wrong.
Working on Notesnook, I have realized no one wants a simple solution. A 10-page-long feature list is preferred over something that does one thing and does it well — the Unix Philosophy, so to say.
No. I had no time to deal with a new tool that'd break in 100 different ways. I wanted a solution that worked with our existing codebase without too many changes, so I did what I should have done from the beginning:
I wrote a script.
144 lines (including blank lines & comments) and only 3 package dependencies (
listr) for nicer DX, doing only one thing: bootstrapping. Here's how it looks:
What's the benefit? Why go to all this effort? Did it really pay off?
- Our CI run time decreased by 1 minute per job across the board. That sounds little, but it all adds up when that job runs 60 times a day.
- We have yet to delete the
node_modulesdirectory even once, and it has been a month, almost.
- We can do
npm ianywhere we want without breaking anything.
- The perception of speed when running
npm run bootstrapmakes us feel 10x faster.
- Lockfiles never get outdated or suddenly updated for no reason.
- It's just
npm idoing all the work, so everything works without changes.
Best of all, there's not been one occasion where any of us has screamed with frustration over Lerna getting stuck.
Life is simple again.
Bootstrapping a monorepo is a surprisingly simple task consisting of only 2 steps:
- Find all the locally linked dependencies of all the subprojects recursively.
- Go into each linked dependency and run
A bash wizard could probably do this in 3 lines. I won't put here all the code because it's too obvious to warrant a discussion, but you can have a look at the code here.
This is our story. Don't take it as a pitch against using Lerna or any other tool. Lerna serves many other usecases which I haven't (yet) found a need for.
In the end, use whatever works best for you and when that breaks look for the simplest solution instead of the "industry solution" or "trends". After all, the only thing a tool is meant to do is save your time.