Example Bazel Monorepo

Note: Currently supporting the latest Bazel version as at mid June 2021, 4.1.0

Example Bazel-ified monorepo, supporting Golang, Java, Python, Scala, and Typescript.

Cloud Infrastructure-as-Code is done using Terraform.

I use this project to explore how Bazel works with different languages and developer tools, and keep a record of best-practices I've learnt. So it is a work in progress. Others can use it to check out the Bazel way of doing things and use parts as a reference implementation.

Rather than the typical To-Do list, this project's code uses the contrived scenario of a book shop and reading catalogue website called Antilibrary. 📗📕📒📚

Getting Started

Prerequisites:

• Install Bazel (Currently supporting ~= 4.x.x)
• Python 2 or 3. Should only be required to do some bootstrapping under-the-hood.
• yarn or npm for the NodeJS and Typescript code

Bazel aims to be 'build anything, anywhere' system, so building and testing should be as simple as bazel test //.... If it's not, please create an issue.

Why use a Monorepo?

The following few articles together provide a good overview of the motivations behind maintaining a Monorepo. For heaps more information, korfuri/awesome-monorepo is a good place to go.

• Why Google Stores Billions of Lines in a Single Repository
• Monorepos, please do!, by Adam Jacob, former CTO of Chef
• Repo Style Wars: Mono vs. Multi, by Peter Seibel
• Advantages of Monorepos, by Dan Luu

Related Projects

• github.com/lucperkins/colossus - A demo using Bazel in monorepo fashion. Compared with this project, it goes far deeper on microservice architecture components and Kubernetes, and is not focused on Bazel.
• github.com/enginoid/monorepo-base - Employs Bazel, gRPC, and Kubernetes like the above, and is similarly not as broad and deep on Bazel as this project.

Project Structure

Golang Support

There's Golang code in /cli. It implements a simple CLI for the common 'Blind Date With a 📖' product.

Dependency Management

Third-party dependencies are managed in 3rdparty/go_workspace.bzl.

Java Support

There's a Spring Boot (with PostGres) application in /store-api and some other Java code in /store/layoutsolver.

Dependency Management

Its third-party dependencies are managed by rules_jvm_external in the WORKSPACE (See the # JAVA SUPPORT section).

Scala Support

There's Scala code contained in scala-book-sorting.

Dependency Management

Its third-party dependencies are managed by johnynek/bazel-deps. The usage of that tool is wrapped up in a script as tools/update_jvm_dependencies.sh.

To use it, you update tools/dependencies/jvm_dependencies.yaml and then run the script.

Python Support

There's Python code in the /book_sorting and /scraping.

bazelbuild/rules_python is used for the core py_* rules.

Dependency Management

In order to add new third-party packages for Python, add them to 3rdparty/requirements.in and run bazel run //3rdparty:requirements.update.

Gradual Type-Checking (MyPy)

thundergolfer/bazel-mypy-integration is used to check any type annotations at bazel build time.

Infrastructure-as-Code (Hashicorp Terraform)

The infrastructure/ top-level folder contains Terraform defining various AWS resources and their configuration.

Development

Build

bazel build //...

Testing

bazel test //...

Continuous Integration (CI)

This repository's CI is managed by Buildkite, the CI platform used by Pinterest and Canva to manage Bazel monorepos, as well as being used by the Bazel open-source project itself.

Deployment & Distribution

Deployable artifacts are pushed to S3 under commit-hash-versioned keys. Currently only the store-api deploy/fat JAR is deployable.

graknlabs/bazel-distribution is used to publish Python packages to PyPi.

Build Observability + Analysis

This project is using Buildbuddy.IO. Every build run locally or in CI get its own https://app.buildbuddy.io/invocation/xyz123... URL which analyses and records the build's information.

Linting

thundergolfer/bazel-linting-system is used. ./tools/linting/lint.sh will lint all source-code in the repo and ./tools/linting/lint_bzl_files.sh will lint all Bazel files.