The lockfile feature in Bazel enables the recording of specific versions or dependencies of software libraries or packages required by a project. It achieves this by storing the result of module resolution and extension evaluation. The lockfile promotes reproducible builds, ensuring consistent development environments. Additionally, it enhances build efficiency by allowing Bazel to skip the parts of the resolution process that are unaffected by changes in project dependencies. Furthermore, the lockfile improves stability by preventing unexpected updates or breaking changes in external libraries, thereby reducing the risk of introducing bugs.
Lockfile Generation
The lockfile is generated under the workspace root with the name
MODULE.bazel.lock
. It is created or updated during the build process,
specifically after module resolution and extension evaluation. Importantly, it
only includes dependencies that are included in the current invocation of the
build.
When changes occur in the project that affect its dependencies, the lockfile is automatically updated to reflect the new state. This ensures that the lockfile remains focused on the specific set of dependencies required for the current build, providing an accurate representation of the project's resolved dependencies.
Lockfile Usage
The lockfile can be controlled by the flag
--lockfile_mode
to
customize the behavior of Bazel when the project state differs from the
lockfile. The available modes are:
update
(Default): Use the information that is present in the lockfile to skip downloads of known registry files and to avoid re-evaluating extensions whose results are still up-to-date. If information is missing, it will be added to the lockfile. In this mode, Bazel also avoids refreshing mutable information, such as yanked versions, for dependencies that haven't changed.refresh
: Likeupdate
, but mutable information is always refreshed when switching to this mode and roughly every hour while in this mode.error
: Likeupdate
, but if any information is missing or out-of-date, Bazel will fail with an error. This mode never changes the lockfile or performs network requests during resolution. Module extensions that marked themselves asreproducible
may still perform network requests, but are expected to always produce the same result.off
: The lockfile is neither checked nor updated.
Lockfile Benefits
The lockfile offers several benefits and can be utilized in various ways:
Reproducible builds. By capturing the specific versions or dependencies of software libraries, the lockfile ensures that builds are reproducible across different environments and over time. Developers can rely on consistent and predictable results when building their projects.
Fast incremental resolutions. The lockfile enables Bazel to avoid downloading registry files that were already used in a previous build. This significantly improves build efficiency, especially in scenarios where resolution can be time-consuming.
Stability and risk reduction. The lockfile helps maintain stability by preventing unexpected updates or breaking changes in external libraries. By locking the dependencies to specific versions, the risk of introducing bugs due to incompatible or untested updates is reduced.
Lockfile Contents
The lockfile contains all the necessary information to determine whether the project state has changed. It also includes the result of building the project in the current state. The lockfile consists of two main parts:
- Hashes of all remote files that are inputs to module resolution.
- For each module extension, the lockfile includes inputs that affect it,
represented by
bzlTransitiveDigest
,usagesDigest
and other fields, as well as the output of running that extension, referred to asgeneratedRepoSpecs
Here is an example that demonstrates the structure of the lockfile, along with explanations for each section:
{
"lockFileVersion": 10,
"registryFileHashes": {
"https://bcr.bazel.build/bazel_registry.json": "8a28e4af...5d5b3497",
"https://bcr.bazel.build/modules/foo/1.0/MODULE.bazel": "7cd0312e...5c96ace2",
"https://bcr.bazel.build/modules/foo/2.0/MODULE.bazel": "70390338... 9fc57589",
"https://bcr.bazel.build/modules/foo/2.0/source.json": "7e3a9adf...170d94ad",
"https://registry.mycorp.com/modules/foo/1.0/MODULE.bazel": "not found",
...
},
"selectedYankedVersions": {
"foo@2.0": "Yanked for demo purposes"
},
"moduleExtensions": {
"//:extension.bzl%lockfile_ext": {
"general": {
"bzlTransitiveDigest": "oWDzxG/aLnyY6Ubrfy....+Jp6maQvEPxn0pBM=",
"usagesDigest": "aLmqbvowmHkkBPve05yyDNGN7oh7QE9kBADr3QIZTZs=",
...,
"generatedRepoSpecs": {
"hello": {
"bzlFile": "@@//:extension.bzl",
...
}
}
}
},
"//:extension.bzl%lockfile_ext2": {
"os:macos": {
"bzlTransitiveDigest": "oWDzxG/aLnyY6Ubrfy....+Jp6maQvEPxn0pBM=",
"usagesDigest": "aLmqbvowmHkkBPve05y....yDNGN7oh7r3QIZTZs=",
...,
"generatedRepoSpecs": {
"hello": {
"bzlFile": "@@//:extension.bzl",
...
}
}
},
"os:linux": {
"bzlTransitiveDigest": "eWDzxG/aLsyY3Ubrto....+Jp4maQvEPxn0pLK=",
"usagesDigest": "aLmqbvowmHkkBPve05y....yDNGN7oh7r3QIZTZs=",
...,
"generatedRepoSpecs": {
"hello": {
"bzlFile": "@@//:extension.bzl",
...
}
}
}
}
}
}
Registry File Hashes
The registryFileHashes
section contains the hashes of all files from
remote registries accessed during module resolution. Since the resolution
algorithm is fully deterministic when given the same inputs and all remote
inputs are hashed, this ensures a fully reproducible resolution result while
avoiding excessive duplication of remote information in the lockfile. Note that
this also requires recording when a particular registry didn't contain a certain
module, but a registry with lower precedence did (see the "not found" entry in
the example). This inherently mutable information can be updated via
bazel mod deps --lockfile_mode=refresh
.
Bazel uses the hashes from the lockfile to look up registry files in the repository cache before downloading them, which speeds up subsequent resolutions.
Selected Yanked Versions
The selectedYankedVersions
section contains the yanked versions of modules
that were selected by module resolution. Since this usually result in an error
when trying to build, this section is only non-empty when yanked versions are
explicitly allowed via --allow_yanked_versions
or
BZLMOD_ALLOW_YANKED_VERSIONS
.
This field is needed since, compared to module files, yanked version information
is inherently mutable and thus can't be referenced by a hash. This information
can be updated via bazel mod deps --lockfile_mode=refresh
.
Module Extensions
The moduleExtensions
section is a map that includes only the extensions used
in the current invocation or previously invoked, while excluding any extensions
that are no longer utilized. In other words, if an extension is not being used
anymore across the dependency graph, it is removed from the moduleExtensions
map.
If an extension is independent of the operating system or architecture type, this section features only a single "general" entry. Otherwise, multiple entries are included, named after the OS, architecture, or both, with each corresponding to the result of evaluating the extension on those specifics.
Each entry in the extension map corresponds to a used extension and is identified by its containing file and name. The corresponding value for each entry contains the relevant information associated with that extension:
- The
bzlTransitiveDigest
is the digest of the extension implementation and the .bzl files transitively loaded by it. - The
usagesDigest
is the digest of the usages of the extension in the dependency graph, which includes all tags. - Further unspecified fields that track other inputs to the extension, such as contents of files or directories it reads or environment variables it uses.
- The
generatedRepoSpecs
encode the repositories created by the extension with the current input. - The optional
moduleExtensionMetadata
field contains metadata provided by the extension such as whether certain repositories it created should be imported viause_repo
by the root module. This information powers thebazel mod tidy
command.
Module extensions can opt out of being included in the lockfile by setting the
returning metadata with reproducible = True
. By doing so, they promise that
they will always create the same repositories when given the same inputs.
Best Practices
To maximize the benefits of the lockfile feature, consider the following best practices:
Regularly update the lockfile to reflect changes in project dependencies or configuration. This ensures that subsequent builds are based on the most up-to-date and accurate set of dependencies. To lock down all extensions at once, run
bazel mod deps --lockfile_mode=update
.Include the lockfile in version control to facilitate collaboration and ensure that all team members have access to the same lockfile, promoting consistent development environments across the project.
Use
bazelisk
to run Bazel, and include a.bazelversion
file in version control that specifies the Bazel version corresponding to the lockfile. Because Bazel itself is a dependency of your build, the lockfile is specific to the Bazel version, and will change even between backwards compatible Bazel releases. Usingbazelisk
ensures that all developers are using a Bazel version that matches the lockfile.
By following these best practices, you can effectively utilize the lockfile feature in Bazel, leading to more efficient, reliable, and collaborative software development workflows.
Merge Conflicts
The lockfile format is designed to minimize merge conflicts, but they can still happen.
Automatic Resolution
Bazel provides a custom git merge driver to help resolve these conflicts automatically.
Set up the driver by adding this line to a .gitattributes
file in the root of
your git repository:
# A custom merge driver for the Bazel lockfile.
# https://bazel.build/external/lockfile#automatic-resolution
MODULE.bazel.lock merge=bazel-lockfile-merge
Then each developer who wants to use the driver has to register it once by following these steps:
- Install jq (1.5 or higher).
- Run the following commands:
jq_script=$(curl https://raw.githubusercontent.com/bazelbuild/bazel/master/scripts/bazel-lockfile-merge.jq)
printf '%s\n' "${jq_script}" | less # to optionally inspect the jq script
git config --global merge.bazel-lockfile-merge.name "Merge driver for the Bazel lockfile (MODULE.bazel.lock)"
git config --global merge.bazel-lockfile-merge.driver "jq -s '${jq_script}' -- %O %A %B > %A.jq_tmp && mv %A.jq_tmp %A"
Manual Resolution
Simple merge conflicts in the registryFileHashes
and selectedYankedVersions
fields can be safely resolved by keeping all the entries from both sides of the
conflict.
Other types of merge conflicts should not be resolved manually. Instead:
- Restore the previous state of the lockfile
via
git reset MODULE.bazel.lock && git checkout MODULE.bazel.lock
. - Resolve any conflicts in the
MODULE.bazel
file. - Run
bazel mod deps
to update the lockfile.