Introduction to TurboRepo

Intro

This article is about turborepo, a smart build system backed by Vercel, It solves many challenges when we try to develop in a monorepo.

Turborepo is a smart build system that addresses many pain points when we develop in a plain monorepo, it brings a lot of benefits and solutions to the challenges we face when developing in a monorepo.

Monorepo

Before diving into turborepo, let’s briefly discuss about monorepo. So what is a monorepo

Monorepo is simply putting multiple projects together into a single repository

Advantages of Monorepo

1. Code Sharing

We no longer have to publish internal UI, and util libraries to npm, we will have direct access to those packages in our mono repo.

2. Code visibility

All of the projects are aggregated into one repository, we can easily view all other projects’ code instead of navigating to different repositories.

3. Integrating guidelines easily (coding style, linting, formatting)

We can now share all the config files like eslint, prettierrc in a single repository.

Challenges of Monorepo

1. Broken master branch

Whenever the master branch is broken, it would possibly affect projects.

2. Longer build time

Since all the projects are centralized in a single repository, the build time would significantly increase if we do not do any optimization

and much more…

As we can see monorepo faces many challenges and many smart build systems come with great ideas for solving these issues, the most popular one are Nrwl NX, Turbo and Rush etc. We are going to talk about turborepo next.

Turborepo

Let’s begin

• Execute npx create-turbo@latest
• During creation, the CLI will ask you to choose your package manager

The folder structure looks like this after successfully created

Folder Structure

apps

apps are where your deployed web projects live.

packages

Packages are where shared libraries like UI, and utils live which can be shared in the apps directory.

Understanding imports and exports

So how do we import the shared library into our projects ? Here are the steps.

1. Specifying the properties in the shared libraries package.json

name → the package name imported by the project

main → the entry of the exported package

packages/ui/package.json

2. Inside a project in the apps directory

Now, you simply add the package name you declare above to the dependencies

importing tsconfig

Inside the `package.json` file in the tsconfig workspace, three files are exported.

You can then add it to other workpsace’s devDependencies, the package name also corresponds to the name property.

You can then extend your tsconfig file inside the project

Cache

Turborepo supports two kinds of caching, local caching and remote caching

local caching

According to the official docs, turbo evaluates the input files of your task and generates a hash, the local cache stays under .cache/turbo .

When you run the task again without changing any input, turbo will try to find the result in the cache first. The result will show how many cache hits and if hits 100% will show a FULL TURBO sign.

With local caching, you no longer have to wait the same command with the same input files twice.

remote caching

When multiple developers are working in a same monorepo, remote caching might be beneficial and here is how it works.

Now all the developers are going to have two caches, the local cache and remote cache. Once a developer runs a command, turbo will first check if there are caches in the remote cache first.

This would benefit if we run duplicate command during our build process. For example, most of the projects will setup pre-commit hooks and continuous integration which runs commands like npm run lint or npm run test targeting the same files.

With remote caching, when pre-commit hooks finishes running these commands, it would save the result to the remote cache. When the command gets triggered again in our CI process, we could pull the result from our remote cache to save our time.

Task Management

Turborepo allows you to leverage multiple cores in your computer to deal with tasks at the same time.

Turborepo allows you to multi-task if you give it enough information to understand dependencies.

from https://turbo.build/repo/docs/core-concepts/monorepos/running-tasks

How it works

Define a pipeline for turbo to understand the order of the tasks to execute

{
  "$schema": "https://turborepo.org/schema.json",
  "pipeline": {
    "build": {
+      // A workspace's `build` task depends on that workspace's
+      // topological dependencies' and devDependencies'
+      // `build` tasks  being completed first. The `^` symbol
+      // indicates an upstream dependency.
      "dependsOn": ["^build"]
    },
    "test": {
+      // A workspace's `test` task depends on that workspace's
+      // own `build` task being completed first.
      "dependsOn": ["build"],
      "outputs": [],
+      // A workspace's `test` task should only be rerun when
+      // either a `.tsx` or `.ts` file has changed.
      "inputs": ["src/**/*.tsx", "src/**/*.ts", "test/**/*.ts", "test/**/*.tsx"]
    },
    "lint": {
+      // A workspace's `lint` task has no dependencies and
+      // can be run whenever.
      "outputs": []
    },
    "deploy": {
+      // A workspace's `deploy` task depends on the `build`,
+      // `test`, and `lint` tasks of the same workspace
+      // being completed.
      "dependsOn": ["build", "test", "lint"],
      "outputs": []
    }
  }
}

Let’s look at the first pipeline

"build": {
+      // A workspace's `build` task depends on that workspace's
+      // topological dependencies' and devDependencies'
+      // `build` tasks  being completed first. The `^` symbol
+      // indicates an upstream dependency.
      "dependsOn": ["^build"]
    },

The ^build means a workspace’s build depends on its dependency and devDependencies build, so if A workspace has B workspace as it’s dependency, A’s build command will only run after B’s build command finishes

Here is a rough timeline indicator

app A:                [building A...]
app B: [building B...]

Let’s look at the last pipeline

"deploy": {
+      // A workspace's `deploy` task depends on the `build`,
+      // `test`, and `lint` tasks of the same workspace
+      // being completed.
      "dependsOn": ["build", "test", "lint"],
      "outputs": []
    }

• the command without a ^ sign specifies each workspace’s deploy depends on each workspace’s ownbuild test lintcommand, which will execute those command first before executing deploy

How it benefits

Traditionally using tools like yarn we can only run one task at a time.

Now, Turbo allows you to multi-task tasks which might speed up the development and build process.

Filtering by workspaces

We might have many workspaces in our turborepo, hence turbo gives us the option to run a command in a single workspace instead targeting all the workspaces

How it works

Giving the --filter argument to match the workspaces

• Filter by a single workspace

turbo run test --filter=ui

• Filter by a workspace’s depedency

# Test 'my-lib' and everything that depends on 'my-lib'
turbo run test --filter=...my-lib
 
# Test everything that depends on 'my-lib', but not 'my-lib' itself
turbo run test --filter=...^my-lib

• Filter by the most recent commit

turbo run test --filter=[HEAD^1]

and much more patterns inside the official doc.

How it benefits

From my point of view I think filter by the most recent commit would benefit us from deploying.

If we setup CI/CD for deploying to an environment once the master branch changes, we could run the command like below to only deploy the related changed projects in the workspace.

turbo run deploy --filter=[HEAD^1]

Conclusion

There will be pros and cons when applying monorepo to our projects. Smart build systems helps us solve a portion of these challenges which is awesome.

Comparing to Nx, turborepo is still young, but I believe it would definitely shine in the future.