Managing monorepos with Lerna and Yarn workspaces

Lerna and Yarn Workspaces gives us the ability to build and publish libraries and apps in a single repository (a.k.a. Monorepo). In this article we'll see how to publish UI components from a monorepo.

Yomi Eluwande
February 1, 2022
Monorepo
Lerna
Yarn
Npm
React

Monorepos are a great way of building multi-package apps as they help keep all of your code in one single place. Some popular examples of monorepos are Next.js, React, Gatsby and Babel.

For Parca, we have our frontend directory set up to be a monorepo. The Parca frontend monorepo currently contains:

  • An app folder that currently houses only the web app (the actual Parca UI).
  • A shared folder that contains packages, functions, utils, and any other thing that can be reusable within apps and even packages.

The monorepo was initialized and maintained by using Yarn Workspaces.

Yarn Workspaces helps to create and maintain a structure where every package/app has its package.json file but at the same time reduces bloat by hoisting all shared dependencies and installing them at the root of your project. It can also identify internal dependencies (for example, two packages in the shared folder depend on each other) and symlink them to make local development seamless.

Lerna is a tool that optimizes the workflow around managing multi-package repositories with git by providing high-level commands. It also allows you to publish packages to any registry.

Using Lerna, you can iterate through all the packages, run a series of operations (such as linting, testing, and building) on each package. In a way, it is very similar to Yarn Workspaces which makes them a good combination for managing and publishing UI components.

Why are we publishing components?

We’re currently hard at work launching the open beta at polarsignals.com and we’d like to reuse some of the components in the open-source Parca project without rewriting code from scratch.

Publishing the UI components to a registry helps solve that problem because these UI components can then be easily used in other projects just by downloading them from the (npm) registry.

Initializing Lerna

We already have Yarn Workspaces set up in the Parca repository, so we will not be looking at how to set up one in this article. This is an excellent and recommended article on setting up a monorepo using Yarn Workspaces.

As you can see here we already have our workspaces defined in the package.json:

  • packages/app/* - this is where to find all our apps, including the UI for Parca
  • packages/shared/* - this is where to find all our shared libraries and functions.

The packages/shared/* directory is what we’re interested in publishing. It contains the code for parsing data, profile viewing libraries, utility functions e.t.c

To initialize Lerna, we first need to add lerna to the project, using the command below.

yarn add lerna --dev

Then we initialize lerna by running the lerna init command. That creates a lerna.json file at the root of the project, which can be edited and replaced with the code block below.

{
  "version": "0.0.0",
  "npmClient": "yarn",
  "useWorkspaces": true,
  "packages": ["packages/shared/*"],
  "command": {
    "publish": {
      "conventionalCommits": true
    },
    "version": {
      "allowBranch": "main"
    }
  }
}

The version has been explicitly set to a version number and this means that all the packages in our monorepo will share this version number. Setting it to independent will mean maintaining the version of each package separately.

npmClient is used to specify what type of package client to run commands with. Defaults to "npm".

The useWorkspaces flag is what will allow us to reuse the yarn workspace settings for lerna.

packages is an array of globs to use as package locations. This should have been exactly similar to the Yarn Workspaces setup, however, we omitted packages/app/* because we don't intend on publishing our apps, only the shared libraries.

The conventionalCommits option when enabled helps to populate a CHANGELOG.md with changes every time there’s a new release. At Parca, we already follow the Conventional Commits style as seen in our contributing guide.

command.version.allowbranch is an option that ensures that publishing only happens on a specified git branch. In our case, it’s the main branch.

Check out Lerna’s GitHub page to see more documentation about the lerna.json file.

There are two other things we need to do before we’re done with setting up Lerna. First, because we don’t want to publish the content of the packages/app/* directory, we need to add the line of code below to every package.json file. This is because Lerna will never publish if a repository is private.

"private":true

The other thing we need to do is modify the `package.json` file for every package in the `packages/shared/*` directory to have these lines of code below.

"publishConfig": {
  "access": "public",
  "registry": "https://registry.npmjs.org/"
}

Lastly, let’s also modify the `package.json` file at the root of the project to have these new two scripts:

{
  “scripts”: {
    …
    "bootstrap": "lerna bootstrap",
    "publish:ci": "lerna publish --yes --no-verify-access",
   }
}

These two commands will be responsible for bootstrapping the dependencies in the monorepo and publishing them to npm. The bootstrap command links the local packages together and installs other dependencies.

Automate Publishing with GitHub Actions

Now that Lerna has been set up successfully, let us see how to publish the UI components as packages to the npm registry.

We’ll need to automate this so that whenever there’s a merge into the main branch, lerna publishes a new version of the appropriate component that was modified.

Using GitHub Actions, we’ll set up a new workflow that will trigger on every "Pull Request merged" event against the main branch.

We need to create a new file, publish.yml in the .github/workflows directory, and edit it with the code block below.

name: Publish


on:
  pull_request:
    types: [closed]
    branches: [main]


jobs:
  publish-ui-components:
    name: Publish UI components to NPM
    runs-on: ubuntu-latest
    env:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    strategy:
      matrix:
        node-version: [14.x]
    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: '0'


      - name: Pull all tags for Lerna semantic release
        run: git fetch --depth=1 origin +refs/tags/*:refs/tags/*


      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v2.5.1
        with:
          node-version: ${{ matrix.node-version }}


      - name: Ensure access
        working-directory: ui
        run: |
          npm config set '//registry.npmjs.org/:_authToken' "${NPM_TOKEN}"
        env:
          NPM_TOKEN: ${{ secrets.NPMTOKEN }}


      - name: Config git user
        run: |
          git config --global user.name "${{ github.actor }}"
          git config --global user.email "${{ github.actor }}@users.noreply.github.com"


      - name: Bootstrap lerna
        working-directory: ui
        run: yarn bootstrap


      - name: Bump versions and publish packages
        working-directory: ui
        run: yarn publish:ci

Let us go over some of the key steps in the workflow file to get a sense of what’s happening.

We first start by naming the workflow Publish and adding an on event listener that will trigger this workflow. As previously stated, we only want to publish the components when there's an update so we set it to only anytime a PR is merged into the main branch.

According to this open issue, there seems to be a bug with using Lerna in GitHub actions. ​Lerna will always create an incorrect version bump when used in GitHub Action because by default the action Checkout V2 (which we use in the workflow) will only fetch the repository with --depth=1, and that means fetching incomplete tags.

This is why we have fetch-depth: "0" and a step called, Pull all tags for Lerna semantic release in the workflow file.

To publish components to NPM, GitHub Actions needs a way to authenticate with npm and that’s what the “Ensure access” step is for. We create a token on npm and use the GitHub secrets to store the token.

The “Bootstrap lerna” step is used to link the local packages together and install other dependencies, while the final step, “Bump versions and publish packages”, uses the publish:ci script to check for updates to any of the components, bumps up the version and publishes to npm.

And that’s all that’s needed to publish the UI components! Here’s a recent action in which the components were updated and published to the npm registry, but you can already see screenshots below of all successfully published packages.

The complete PR for adding Lerna to the Parca project is also available on GitHub.

If you have any questions or suggestions on how we manage our UI components, feel free to join our Discord community server. You can also star the Parca project on GitHub and contributions are welcome!

Discuss:
Sign up for the latest Polar Signals news