| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Breaks Javascript compilation.
There's currently no way to iterate over a depset of Artifacts and deduplicate by identical paths in Skylark. This means that actions that want to do something once per Artifact in a depset (add a flag to the command line with the path of the Artifact for instance) will have duplicate entries if there are multiple Artifacts with the same path.
This is not a true automated rollback, since I tried to make this as minimal as possible, to avoid merge conflicts and keep some of the benefits of the rolled back CL. In particular, the tests that were changed in the original CL to give artifacts their correct owners did not need to be changed back, since the owners are still correct.
Moreover, this effectively contains rollbacks of unknown commit and https://github.com/bazelbuild/bazel/commit/39d6f89644107a8f7c080c4f4aec8527c1a73954, but keeps test coverage from those CLs as well. Comments added to CL thread where not a clean rollback (there are plenty of files not changed at all: ActionArtifactCycleReporter is the main wart, since it can still handle SkyKeys with Artifact as the argument, but Artifact is no longer the argument to any SkyKey).
RELNOTES: None
*** Original change description ***
Make Artifact#equals take the owner into account for derived artifacts.
Derived artifacts' owners are important because they are used to determine the artifact's generating action. Source artifacts' owners are not used in this way, so I left them alone.
This allows us to get rid of most uses of ArtifactSkyKey. We may be able to delete it entirely in a follow-up.
PiperOrigin-RevId: 201464780
|
|
|
|
|
|
|
|
| |
ActionConflictException. We can construct the exception message eagerly, since it is going to be thrown with very high likelihood in any case.
The complex objects inside it made it hard to serialize.
PiperOrigin-RevId: 201130522
|
|
|
|
|
|
|
| |
The memory savings from this flag are not worth the complexity, and it interferes with action restarting.
RELNOTES: Remove support for --discard_actions_after_execution.
PiperOrigin-RevId: 201077905
|
|
|
|
|
|
| |
Moves more Action-oriented from skyframe package, which has become very unwieldy, to action package. This is needed to avoid circular dependencies caused when build-base is needed for skyframe.
PiperOrigin-RevId: 200996982
|
|
|
|
|
|
|
|
| |
This should be a no-op, mostly replacing PathConverter with
BuildEventArtifactUploader, since none of the implementations perform any
upload yet.
PiperOrigin-RevId: 200685325
|
|
|
|
|
|
| |
not serialized. Tag TestCompletionValue and any ActionExecutionValue coming from a NotifyOnActionCacheHit (i.e., tests) like that. To make such values really not shared, request the ActionExecutionValue from TestCompletionFunction as opposed to the ArtifactValue (propagating the unshareable bit up seemed like too much fuss, and I have a dream of getting rid of ArtifactValue anyway).
PiperOrigin-RevId: 200504358
|
|
|
|
|
|
|
| |
stdout/stderr.
RELNOTES: None
PiperOrigin-RevId: 200459354
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 199880252
|
|
|
|
|
|
|
|
| |
Derived artifacts' owners are important because they are used to determine the artifact's generating action. Source artifacts' owners are not used in this way, so I left them alone.
This allows us to get rid of most uses of ArtifactSkyKey. We may be able to delete it entirely in a follow-up.
PiperOrigin-RevId: 199836436
|
|
|
|
| |
PiperOrigin-RevId: 199820207
|
|
|
|
|
|
|
|
|
| |
When a LATIN-1 parameter file is requested, we can take advantage of the fact that JDK9 strings are (usually) stored as LATIN-1.
For UTF-8, we can still optimize for the common case where a LATIN-1 string contains only ASCII characters, as these are bit-identical between UTF-8 and LATIN-1. This would still be expected to be the vast majority of parameter file contents.
RELNOTES: None
PiperOrigin-RevId: 199816430
|
|
|
|
|
|
| |
Small misc cleanups.
PiperOrigin-RevId: 199797948
|
|
|
|
| |
PiperOrigin-RevId: 199732415
|
|
|
|
|
|
| |
(minor) ActionFS now implements MetadataProvider.getInput
PiperOrigin-RevId: 199575194
|
|
|
|
|
|
| |
Adds a helper class, ActionInputMap to do this with minimal wrapping overhead.
PiperOrigin-RevId: 199391251
|
|
|
|
|
|
| |
Actual class to be removed in a later change.
PiperOrigin-RevId: 198937695
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
classes are not equal.
Also validate that there are no tree and file artifacts with the same exec path.
Fixes #4668.
Closes #5284.
Change-Id: Id97c0407a476a5bfc697b4ca7b858e3d0c0f8c75
PiperOrigin-RevId: 198540425
|
|
|
|
|
|
|
| |
Instead,defer to ActionFS regarding the choice about delegation to top-level Filesystem.
RELNOTES: None
PiperOrigin-RevId: 197960416
|
|
|
|
| |
PiperOrigin-RevId: 197924560
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Change introduced flakiness in //src/test/java/com/google/devtools/build/lib:actions_test
Verified the culprit commit using:
bazel test //src/test/java/com/google/devtools/build/lib:actions_test --runs_per_test=100 --notest_keep_going -t-
See https://github.com/bazelbuild/bazel/issues/5242.
*** Original change description ***
Relax the threshold for calling getDigestInExclusiveMode().
If the policy goal is to minimize disk seeks, this should be equally good.
RELNOTES: None.
PiperOrigin-RevId: 197720793
|
|
|
|
|
|
|
|
| |
The constructors which don't accept an action environment should not be called,
with the exception of subclasses that don't run any subprocesses, and therefore
don't need an environment.
PiperOrigin-RevId: 197706734
|
|
|
|
|
|
|
| |
If the policy goal is to minimize disk seeks, this should be equally good.
RELNOTES: None.
PiperOrigin-RevId: 197611813
|
|
|
|
|
|
|
|
| |
Instead, internally look up the correct context by mnemonic. This simplifies
all the callers. We still need a little bit of special casing when constructing
the action context map.
PiperOrigin-RevId: 197572357
|
|
|
|
| |
PiperOrigin-RevId: 197545650
|
|
|
|
|
|
|
| |
This is in preparation for adding options to the class, which can be used
to modify the behavior of build events posted to the BEP.
PiperOrigin-RevId: 196997573
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a cleanup to clarify the code.
1. The getEnvironment method in the CommandAction interface does not have
access to the clientEnv, so it's return value is necessarily incomplete.
Rename to getIncompleteEnvironmentForTesting.
2. Add a final getEnvironment method to AbstractAction, which returns the
ActionEnvironment, which is intended to be a complete description of the
intended final environment of the action (technically, of any spawn running
within the action). This is not currently used, but is provided to prevent
action subclasses to add such a method (it may be used in the future).
PiperOrigin-RevId: 196991091
|
|
|
|
|
|
|
|
|
|
| |
These were previously ignoring the inhertied environment, i.e.,
--action_env=PATH did _not_ result in the PATH variable being forwarded from
the client environment.
Fixes #5142.
PiperOrigin-RevId: 196966822
|
|
|
|
|
|
|
| |
This is in preparation for fixing env handling as well as cache key (to use
env) computations in subclasses of SpawnAction.
PiperOrigin-RevId: 196626495
|
|
|
|
| |
PiperOrigin-RevId: 196290095
|
|
|
|
| |
PiperOrigin-RevId: 195973862
|
|
|
|
|
|
| |
contention and use a striped lock.
PiperOrigin-RevId: 195717688
|
|
|
|
|
|
|
|
|
|
|
|
| |
This effectively subsumes all existing memory estimation, as it no
longer matters too much how much memory is assigned to each action.
WIP: This is a hack, and if it goes in, should almost certainly be
hidden behind a configuration option.
Closes #4938.
PiperOrigin-RevId: 195378568
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 195280015
|
|
|
|
|
|
|
| |
instead of the manifest files.
RELNOTES: None
PiperOrigin-RevId: 195149880
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 195011200
|
|
|
|
| |
PiperOrigin-RevId: 194985157
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 194960076
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 194798051
|
|
|
|
| |
PiperOrigin-RevId: 194413337
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In StandaloneTestStrategy, copy as much information as SpawnResult makes
available to us through to both the TestResultData and BEP's
TestResult.ExecutionInfo protos. One immediate consequence is that the UI and
BEP can tell you whether a test result was cached remotely.
I changed Executor.getEventHandler to return an ExtendedEventHandler because it
makes this change easier to test.
Closes #5081.
Change-Id: I94fefdcd2e029c81085076736ad13a4bdf1bae8f
PiperOrigin-RevId: 194383009
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Design doc: https://docs.google.com/document/d/1JXqwwVHYosZOgmjN8xrfTalyhiUYJ99Qe2D0qBcqZ1c
The behaviour is gated on --defer_param_files (default off) and is controlled by --min_param_file_size.
This CL adds support for VirtualActionInputs to LocalSpawnRunner, and all remote runners already supports them. The sandboxed runners are not yet supported, but that can be added in a future CL.
This CL does not add support for spawn runner using different param file limits. This will require refactoring of the spawn strategies and runners to be viable.
RELNOTES: None
PiperOrigin-RevId: 194265291
|
|
|
|
|
|
| |
This should reduce memory consumption in NestedSet deserialization, which currently does not recycle Artifact instances.
PiperOrigin-RevId: 194083901
|
|
|
|
|
|
|
|
|
| |
We're mostly adding convenience methods. This will be used by spawn runners and in Spawn construction, but we break it out now to lessen the size of the final CL.
We also pull out a CommandLineLimits class instead of using an int. This will partially help with readability, but it also likely that we will add a second constraint (max length of a single argument).
RELNOTES: None
PiperOrigin-RevId: 193191097
|
|
|
|
| |
PiperOrigin-RevId: 193130164
|
|
|
|
|
|
|
| |
Since we're only supporting a single %s anyway, make this explicit in the docs, fail in the analysis phase, and use a more efficient format method in the implementation.
RELNOTES: None
PiperOrigin-RevId: 193099585
|
|
|
|
|
|
|
| |
When going ahead with the implementation, the full name felt too verbose. CommandLines still shows that we have a list of command lines, and the attached param file info is secondary enough that I'm comfortable omitting the name.
RELNOTES: None
PiperOrigin-RevId: 192731592
|
|
|
|
|
|
| |
RELNOTES: Args objects (ctx.actions.args()) have new methods add_all() and add_joined() for building command lines using depsets.
RELNOTES[INC]: Added flag --incompatible_disallow_old_style_args_add to help migrate from args.add() to args.add_all() / args.add_joined() where appropriate.
PiperOrigin-RevId: 192633763
|
|
|
|
| |
PiperOrigin-RevId: 192628723
|
|
|
|
|
|
|
|
| |
the ones in SkyframeExecutor, called once for each Root in the package path list. This ensures there is a single canonical ArtifactRoot for each source root.
It is still the case that every Package loaded has its own Root (https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/packages/Package.java;l=287?q=Package.java), which is a waste of memory, but because of the map inside ArtifactFactory introduced in this change, all Artifacts with the same logical source root share a single ArtifactRoot instance.
PiperOrigin-RevId: 192513819
|
|
|
|
|
| |
RELNOTES: Exposed "mnemonic" and "env" fields on skylark "Action" objects.
PiperOrigin-RevId: 192479783
|