Intro
This article is about turborepo, a smart build system backed by Vercel, It solves many challenges when we try to develop in a monorepo.
Turborepo is a smart build system that addresses many pain points when we develop in a plain monorepo, it brings a lot of benefits and solutions to the challenges we face when developing in a monorepo.
Monorepo
Before diving into turborepo, let’s briefly discuss about monorepo. So what is a monorepo
Monorepo is simply putting multiple projects together into a single repository
Advantages of Monorepo
- Code Sharing
We no longer have to publish internal UI, and util libraries to npm, we will have direct access to those packages in our mono repo.
2. Code visibility
All of the projects are aggregated into one repository, we can easily view all other projects’ code instead of navigating to different repositories.
3. Integrating guidelines easily (coding style, linting, formatting)
We can now share all the config files like eslint, prettierrc in a single repository.
Challenges of Monorepo
- Broken master branch
Whenever the master branch is broken, it would possibly affect projects.
2. Longer build time
Since all the projects are centralized in a single repository, the build time would significantly increase if we do not do any optimization
and much more…
As we can see monorepo faces many challenges and many smart build systems come with great ideas for solving these issues, the most popular one are Nrwl NX, Turbo and Rush etc. We are going to talk about turborepo next.
Turborepo
Let’s begin
- Execute
npx create-turbo@latest
- During creation, the CLI will ask you to choose your package manager
The folder structure looks like this after successfully created
Folder Structure
apps
apps are where your deployed web projects live.
packages
Packages are where shared libraries like UI, and utils live which can be shared in the apps directory.
Understanding imports and exports
So how do we import the shared library into our projects ? Here are the steps.
- Specifying the properties in the shared libraries
package.json
name → the package name imported by the project
main → the entry of the exported package
2. Inside a project in the apps directory
Now, you simply add the package name you declare above to the dependencies
importing tsconfig
Inside the `package.json` file in the tsconfig workspace, three files are exported.
You can then add it to other workpsace’s devDependencies, the package name also corresponds to the name property.
You can then extend your tsconfig file inside the project
Cache
Turborepo supports two kinds of caching, local caching and remote caching
local caching
According to the official docs, turbo evaluates the input files of your task and generates a hash, the local cache stays under .cache/turbo
.
When you run the task again without changing any input, turbo will try to find the result in the cache first. The result will show how many cache hits and if hits 100% will show a FULL TURBO
sign.
With local caching, you no longer have to wait the same command with the same input files twice.
remote caching
When multiple developers are working in a same monorepo, remote caching might be beneficial and here is how it works.
Now all the developers are going to have two caches, the local cache and remote cache. Once a developer runs a command, turbo will first check if there are caches in the remote cache first.
This would benefit if we run duplicate command during our build process. For example, most of the projects will setup pre-commit hooks and continuous integration which runs commands like npm run lint
or npm run test
targeting the same files.
With remote caching, when pre-commit hooks finishes running these commands, it would save the result to the remote cache. When the command gets triggered again in our CI process, we could pull the result from our remote cache to save our time.
Task Management
Turborepo allows you to leverage multiple cores in your computer to deal with tasks at the same time.
Turborepo allows you to multi-task if you give it enough information to understand dependencies.
How it works
Define a pipeline for turbo to understand the order of the tasks to execute
{
"$schema": "https://turborepo.org/schema.json",
"pipeline": {
"build": {
+ // A workspace's `build` task depends on that workspace's
+ // topological dependencies' and devDependencies'
+ // `build` tasks being completed first. The `^` symbol
+ // indicates an upstream dependency.
"dependsOn": ["^build"]
},
"test": {
+ // A workspace's `test` task depends on that workspace's
+ // own `build` task being completed first.
"dependsOn": ["build"],
"outputs": [],
+ // A workspace's `test` task should only be rerun when
+ // either a `.tsx` or `.ts` file has changed.
"inputs": ["src/**/*.tsx", "src/**/*.ts", "test/**/*.ts", "test/**/*.tsx"]
},
"lint": {
+ // A workspace's `lint` task has no dependencies and
+ // can be run whenever.
"outputs": []
},
"deploy": {
+ // A workspace's `deploy` task depends on the `build`,
+ // `test`, and `lint` tasks of the same workspace
+ // being completed.
"dependsOn": ["build", "test", "lint"],
"outputs": []
}
}
}
Let’s look at the first pipeline
"build": {
+ // A workspace's `build` task depends on that workspace's
+ // topological dependencies' and devDependencies'
+ // `build` tasks being completed first. The `^` symbol
+ // indicates an upstream dependency.
"dependsOn": ["^build"]
},
The ^build
means a workspace’s build depends on its dependency and devDependencies build, so if A workspace has B workspace as it’s dependency, A’s build command will only run after B’s build command finishes
Here is a rough timeline indicator
app A: [building A...]
app B: [building B...]
Let’s look at the last pipeline
"deploy": {
+ // A workspace's `deploy` task depends on the `build`,
+ // `test`, and `lint` tasks of the same workspace
+ // being completed.
"dependsOn": ["build", "test", "lint"],
"outputs": []
}
- the command without a
^
sign specifies each workspace’sdeploy
depends on each workspace’s ownbuild test lint
command, which will execute those command first before executingdeploy
How it benefits
Traditionally using tools like yarn we can only run one task at a time.
Now, Turbo allows you to multi-task tasks which might speed up the development and build process.
Filtering by workspaces
We might have many workspaces in our turborepo, hence turbo gives us the option to run a command in a single workspace instead targeting all the workspaces
How it works
Giving the --filter
argument to match the workspaces
- Filter by a single workspace
turbo run test --filter=ui
- Filter by a workspace’s depedency
# Test 'my-lib' and everything that depends on 'my-lib'
turbo run test --filter=...my-lib
# Test everything that depends on 'my-lib', but not 'my-lib' itself
turbo run test --filter=...^my-lib
- Filter by the most recent commit
turbo run test --filter=[HEAD^1]
and much more patterns inside the official doc.
How it benefits
From my point of view I think filter by the most recent commit would benefit us from deploying.
If we setup CI/CD for deploying to an environment once the master branch changes, we could run the command like below to only deploy the related changed projects in the workspace.
turbo run deploy --filter=[HEAD^1]
Conclusion
There will be pros and cons when applying monorepo to our projects. Smart build systems helps us solve a portion of these challenges which is awesome.
Comparing to Nx, turborepo is still young, but I believe it would definitely shine in the future.