Automatic execution groups select an execution platform for each toolchain type. In other words, one target can have multiple execution platforms without defining execution groups.
Quick summary
Automatic execution groups are closely connected to toolchains. If you are using
toolchains, you need to set them on the affected actions (actions which use an
executable or a tool from a toolchain) by adding toolchain parameter. For
example:
ctx.actions.run(
...,
executable = ctx.toolchain['@bazel_tools//tools/jdk:toolchain_type'].tool,
...,
toolchain = '@bazel_tools//tools/jdk:toolchain_type',
)
If the action does not use a tool or executable from a toolchain, and Blaze
doesn't detect that (the error is raised), you can set
toolchain = None.
If you need to use multiple toolchains on a single execution platform (an action uses executable or tools from two or more toolchains), you need to manually define exec_groups (check When should I use a custom exec_group? section).
History
Before AEGs, the execution platform was selected on a rule level. For example:
my_rule = rule(
_impl,
toolchains = ['//tools:toolchain_type_1', '//tools:toolchain_type_2'],
)
Rule my_rule registers two toolchain types. This means that the Toolchain
Resolution used
to find an execution platform which supports both toolchain types. The selected
execution platform was used for each registered action inside the rule, unless
specified differently with exec_groups.
In other words, all actions inside the rule used to have a single execution
platform even if they used tools from different toolchains (execution platform
is selected for each target). This resulted in failures when there was no
execution platform supporting all toolchains.
Current state
With AEGs, the execution platform is selected for each toolchain type. The
implementation function of the earlier example, my_rule, would look like:
def _impl(ctx):
ctx.actions.run(
mnemonic = "First action",
executable = ctx.toolchain['//tools:toolchain_type_1'].tool,
toolchain = '//tools:toolchain_type_1',
)
ctx.actions.run(
mnemonic = "Second action",
executable = ctx.toolchain['//tools:toolchain_type_2'].tool,
toolchain = '//tools:toolchain_type_2',
)
This rule creates two actions, the First action which uses executable from a
//tools:toolchain_type_1 and the Second action which uses executable from a
//tools:toolchain_type_2. Before AEGs, both of these actions would be executed
on a single execution platform which supports both toolchain types. With AEGs,
by adding the toolchain parameter inside the actions, each action executes on
the execution platform that provides the toolchain. The actions may be executed
on different execution platforms.
The same is effective with ctx.actions.run_shell where toolchain
parameter should be added when tools are from a toolchain.
Difference between custom exec groups and automatic exec groups
As the name suggests, AEGs are exec groups created automatically for each toolchain type registered on a rule. There is no need to manually specify them, unlike the "classic" exec groups.
When should I use a custom exec_group?
Custom exec_groups are needed only in case where multiple toolchains need to execute on a single execution platform. In all other cases there's no need to define custom exec_groups. For example:
def _impl(ctx):
ctx.actions.run(
...,
executable = ctx.toolchain['//tools:toolchain_type_1'].tool,
tools = [ctx.toolchain['//tools:toolchain_type_2'].tool],
exec_group = 'two_toolchains',
)
my_rule = rule(
_impl,
exec_groups = {
"two_toolchains": exec_group(
toolchains = ['//tools:toolchain_type_1', '//tools:toolchain_type_2'],
),
}
)
Migration of AEGs
Internally in google3, Blaze is already using AEGs. Externally for Bazel, migration is in the process. Some rules are already using this feature (e.g. Java and C++ rules).
Which Bazel versions support this migration?
AEGs are fully supported from Bazel 7.
How to enable AEGs?
Set --incompatible_auto_exec_groups to true. More information about the flag
on the GitHub issue.
How to enable AEGs inside a particular rule?
Set the _use_auto_exec_groups attribute on a rule.
my_rule = rule(
_impl,
attrs = {
"_use_auto_exec_groups": attr.bool(default = True),
}
)
This enables AEGs only in my_rule and its actions start using the new logic
when selecting the execution platform. Incompatible flag is overridden with this
attribute.
How to disable AEGs in case of an error?
Set --incompatible_auto_exec_groups to false to completely disable AEGs in
your project (flag's GitHub issue), or disable a particular rule
by setting _use_auto_exec_groups attribute to False
(more details about the attribute).
Error messages while migrating to AEGs
Couldn't identify if tools are from implicit dependencies or a toolchain. Please set the toolchain parameter. If you're not using a toolchain, set it to 'None'.
- In this case you get a stack of calls before the error happened and you can
clearly see which exact action needs the toolchain parameter. Check which
toolchain is used for the action and set it with the toolchain param. If no
toolchain is used inside the action for tools or executable, set it to
None.
Action declared for non-existent toolchain '[toolchain_type]'.
- This means that you've set the toolchain parameter on the action but didn't
register it on the rule. Register the toolchain or set
Noneinside the action.
Additional material
For more information, check design document: Automatic exec groups for toolchains.