aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/actions
Commit message (Collapse)AuthorAge
* Refactoring: uses OutputService for ActionFileSystem injectionGravatar shahan2018-06-18
| | | | | | Moves more Action-oriented from skyframe package, which has become very unwieldy, to action package. This is needed to avoid circular dependencies caused when build-base is needed for skyframe. PiperOrigin-RevId: 200996982
* Add a mechanism for build event protocol events to upload filesGravatar ulfjack2018-06-15
| | | | | | | | This should be a no-op, mostly replacing PathConverter with BuildEventArtifactUploader, since none of the implementations perform any upload yet. PiperOrigin-RevId: 200685325
* Add functionality to make certain SkyValues unshareable, meaning they are ↵Gravatar janakr2018-06-13
| | | | | | not serialized. Tag TestCompletionValue and any ActionExecutionValue coming from a NotifyOnActionCacheHit (i.e., tests) like that. To make such values really not shared, request the ActionExecutionValue from TestCompletionFunction as opposed to the ArtifactValue (propagating the unshareable bit up seemed like too much fuss, and I have a dream of getting rid of ArtifactValue anyway). PiperOrigin-RevId: 200504358
* Pass ActionFS paths through to action-level FileOutErr for the Action's ↵Gravatar felly2018-06-13
| | | | | | | stdout/stderr. RELNOTES: None PiperOrigin-RevId: 200459354
* Unify path resolution codepaths.Gravatar felly2018-06-08
| | | | | RELNOTES: None PiperOrigin-RevId: 199880252
* Make Artifact#equals take the owner into account for derived artifacts.Gravatar janakr2018-06-08
| | | | | | | | Derived artifacts' owners are important because they are used to determine the artifact's generating action. Source artifacts' owners are not used in this way, so I left them alone. This allows us to get rid of most uses of ArtifactSkyKey. We may be able to delete it entirely in a follow-up. PiperOrigin-RevId: 199836436
* ActionFS correctly tracks symlink sources.Gravatar shahan2018-06-08
| | | | PiperOrigin-RevId: 199820207
* Use unsafe String operations when writing parameter files.Gravatar tomlu2018-06-08
| | | | | | | | | When a LATIN-1 parameter file is requested, we can take advantage of the fact that JDK9 strings are (usually) stored as LATIN-1. For UTF-8, we can still optimize for the common case where a LATIN-1 string contains only ASCII characters, as these are bit-identical between UTF-8 and LATIN-1. This would still be expected to be the vast majority of parameter file contents. RELNOTES: None PiperOrigin-RevId: 199816430
* Deletes default implemetation of MetadataProvider.getInputGravatar shahan2018-06-08
| | | | | | Small misc cleanups. PiperOrigin-RevId: 199797948
* Deletes ActionInputFileCache.Gravatar shahan2018-06-07
| | | | PiperOrigin-RevId: 199732415
* Refactoring: makes the code simpler by deleting Metadata and InputArtifactData.Gravatar shahan2018-06-06
| | | | | | (minor) ActionFS now implements MetadataProvider.getInput PiperOrigin-RevId: 199575194
* MetadataProvider now provides ActionInput lookup by exec path.Gravatar shahan2018-06-05
| | | | | | Adds a helper class, ActionInputMap to do this with minimal wrapping overhead. PiperOrigin-RevId: 199391251
* Removes most ActionInputFileCache functionality.Gravatar shahan2018-06-01
| | | | | | Actual class to be removed in a later change. PiperOrigin-RevId: 198937695
* Fix `equals()` and `hashCode()` for artifacts: artifacts of different ↵Gravatar Dmitry Lomov2018-05-30
| | | | | | | | | | | | | classes are not equal. Also validate that there are no tree and file artifacts with the same exec path. Fixes #4668. Closes #5284. Change-Id: Id97c0407a476a5bfc697b4ca7b858e3d0c0f8c75 PiperOrigin-RevId: 198540425
* When in use, always return ActionFS paths from the ActionExecutionContext. ↵Gravatar felly2018-05-24
| | | | | | | Instead,defer to ActionFS regarding the choice about delegation to top-level Filesystem. RELNOTES: None PiperOrigin-RevId: 197960416
* RemoteFileArtifactValue no longer tracks mtime.Gravatar shahan2018-05-24
| | | | PiperOrigin-RevId: 197924560
* Automated rollback of commit 724bdbfa47576c67eeec5c74d594203fe05188c7.Gravatar laszlocsomor2018-05-23
| | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Change introduced flakiness in //src/test/java/com/google/devtools/build/lib:actions_test Verified the culprit commit using: bazel test //src/test/java/com/google/devtools/build/lib:actions_test --runs_per_test=100 --notest_keep_going -t- See https://github.com/bazelbuild/bazel/issues/5242. *** Original change description *** Relax the threshold for calling getDigestInExclusiveMode(). If the policy goal is to minimize disk seeks, this should be equally good. RELNOTES: None. PiperOrigin-RevId: 197720793
* Remove some dead AbstractAction constructorsGravatar ulfjack2018-05-23
| | | | | | | | The constructors which don't accept an action environment should not be called, with the exception of subclasses that don't run any subprocesses, and therefore don't need an environment. PiperOrigin-RevId: 197706734
* Relax the threshold for calling getDigestInExclusiveMode().Gravatar Googler2018-05-22
| | | | | | | If the policy goal is to minimize disk seeks, this should be equally good. RELNOTES: None. PiperOrigin-RevId: 197611813
* Remove special handling of SpawnActionContext in Executor/ActionExecContextGravatar ulfjack2018-05-22
| | | | | | | | Instead, internally look up the correct context by mnemonic. This simplifies all the callers. We still need a little bit of special casing when constructing the action context map. PiperOrigin-RevId: 197572357
* Add a helper method to compute timeouts with a default valueGravatar ulfjack2018-05-22
| | | | PiperOrigin-RevId: 197545650
* Rename BuildEventConverters to BuildEventContextGravatar ulfjack2018-05-17
| | | | | | | This is in preparation for adding options to the class, which can be used to modify the behavior of build events posted to the BEP. PiperOrigin-RevId: 196997573
* Rename CommandAction.getEnvironment, add AbstractAction.getEnvironmentGravatar ulfjack2018-05-17
| | | | | | | | | | | | | | | This is a cleanup to clarify the code. 1. The getEnvironment method in the CommandAction interface does not have access to the clientEnv, so it's return value is necessarily incomplete. Rename to getIncompleteEnvironmentForTesting. 2. Add a final getEnvironment method to AbstractAction, which returns the ActionEnvironment, which is intended to be a complete description of the intended final environment of the action (technically, of any spawn running within the action). This is not currently used, but is provided to prevent action subclasses to add such a method (it may be used in the future). PiperOrigin-RevId: 196991091
* Fix Cpp{Compile,Link}Action environment and cache key computationGravatar ulfjack2018-05-17
| | | | | | | | | | These were previously ignoring the inhertied environment, i.e., --action_env=PATH did _not_ result in the PATH variable being forwarded from the client environment. Fixes #5142. PiperOrigin-RevId: 196966822
* Slightly refactor SpawnAction to improve env handlingGravatar ulfjack2018-05-15
| | | | | | | This is in preparation for fixing env handling as well as cache key (to use env) computations in subclasses of SpawnAction. PiperOrigin-RevId: 196626495
* ActionFS cleanupGravatar shahan2018-05-11
| | | | PiperOrigin-RevId: 196290095
* Adds an action-scoped filesystem.Gravatar shahan2018-05-09
| | | | PiperOrigin-RevId: 195973862
* Narrow synchronized section of ArtifactFactory#getArtifact to avoid ↵Gravatar janakr2018-05-07
| | | | | | contention and use a striped lock. PiperOrigin-RevId: 195717688
* Do not spawn local actions when available memory is too lowGravatar Josh Pieper2018-05-04
| | | | | | | | | | | | This effectively subsumes all existing memory estimation, as it no longer matters too much how much memory is assigned to each action. WIP: This is a hack, and if it goes in, should almost certainly be hidden behind a configuration option. Closes #4938. PiperOrigin-RevId: 195378568
* Use ConcurrentHashMap directly instead of MapMaker.Gravatar Googler2018-05-03
| | | | | RELNOTES: None. PiperOrigin-RevId: 195280015
* Use the in-memory metadata in blaze as the source of truth for Fileset mappingsGravatar kush2018-05-02
| | | | | | | instead of the manifest files. RELNOTES: None PiperOrigin-RevId: 195149880
* Migrate AbstractAction and SkylarkActionFactory to buildapiGravatar cparsons2018-05-01
| | | | | RELNOTES: None. PiperOrigin-RevId: 195011200
* Clean up code that directly imports nested classes like Builder, Entry, etc.Gravatar jcater2018-05-01
| | | | PiperOrigin-RevId: 194985157
* Migrate the skylark interfaces of Artifact and ArtifactRoot to skylarkbuildapiGravatar cparsons2018-05-01
| | | | | RELNOTES: None. PiperOrigin-RevId: 194960076
* Support ParamFileActionInput in LocalSpawnRunner.Gravatar tomlu2018-04-30
| | | | | RELNOTES: None PiperOrigin-RevId: 194798051
* Adds support for remote files.Gravatar shahan2018-04-26
| | | | PiperOrigin-RevId: 194413337
* Add execution information to standalone test results.Gravatar Benjamin Peterson2018-04-26
| | | | | | | | | | | | | | | In StandaloneTestStrategy, copy as much information as SpawnResult makes available to us through to both the TestResultData and BEP's TestResult.ExecutionInfo protos. One immediate consequence is that the UI and BEP can tell you whether a test result was cached remotely. I changed Executor.getEventHandler to return an ExtendedEventHandler because it makes this change easier to test. Closes #5081. Change-Id: I94fefdcd2e029c81085076736ad13a4bdf1bae8f PiperOrigin-RevId: 194383009
* Support deferred param files.Gravatar tomlu2018-04-25
| | | | | | | | | | | | | Design doc: https://docs.google.com/document/d/1JXqwwVHYosZOgmjN8xrfTalyhiUYJ99Qe2D0qBcqZ1c The behaviour is gated on --defer_param_files (default off) and is controlled by --min_param_file_size. This CL adds support for VirtualActionInputs to LocalSpawnRunner, and all remote runners already supports them. The sandboxed runners are not yet supported, but that can be added in a future CL. This CL does not add support for spawn runner using different param file limits. This will require refactoring of the spawn strategies and runners to be viable. RELNOTES: None PiperOrigin-RevId: 194265291
* SourceArtifacts are interned on deserialization using an ArtifactFactory. ↵Gravatar cpeyser2018-04-24
| | | | | | This should reduce memory consumption in NestedSet deserialization, which currently does not recycle Artifact instances. PiperOrigin-RevId: 194083901
* Add helpful functionality to CommandLines.Gravatar tomlu2018-04-17
| | | | | | | | | We're mostly adding convenience methods. This will be used by spawn runners and in Spawn construction, but we break it out now to lessen the size of the final CL. We also pull out a CommandLineLimits class instead of using an int. This will partially help with readability, but it also likely that we will add a second constraint (max length of a single argument). RELNOTES: None PiperOrigin-RevId: 193191097
* Cleanup Artifact.getPath() usage surrounding Fingerprint.Gravatar shahan2018-04-16
| | | | PiperOrigin-RevId: 193130164
* Update format implementation in ctx.actions.args.Gravatar tomlu2018-04-16
| | | | | | | Since we're only supporting a single %s anyway, make this explicit in the docs, fail in the analysis phase, and use a more efficient format method in the implementation. RELNOTES: None PiperOrigin-RevId: 193099585
* Rename CommandLinesAndParamFiles -> CommandLines.Gravatar tomlu2018-04-13
| | | | | | | When going ahead with the implementation, the full name felt too verbose. CommandLines still shows that we have a list of command lines, and the attached param file info is secondary enough that I'm comfortable omitting the name. RELNOTES: None PiperOrigin-RevId: 192731592
* Document the new ctx.actions.args() APIGravatar brandjon2018-04-12
| | | | | | RELNOTES: Args objects (ctx.actions.args()) have new methods add_all() and add_joined() for building command lines using depsets. RELNOTES[INC]: Added flag --incompatible_disallow_old_style_args_add to help migrate from args.add() to args.add_all() / args.add_joined() where appropriate. PiperOrigin-RevId: 192633763
* Plumbs ActionExecutionContext into AbstractAction.checkOutputsForDirectories.Gravatar shahan2018-04-12
| | | | PiperOrigin-RevId: 192628723
* Remove all callers of ArtifactRoot.asSourcePath from production code besides ↵Gravatar janakr2018-04-11
| | | | | | | | the ones in SkyframeExecutor, called once for each Root in the package path list. This ensures there is a single canonical ArtifactRoot for each source root. It is still the case that every Package loaded has its own Root (https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/packages/Package.java;l=287?q=Package.java), which is a waste of memory, but because of the map inside ArtifactFactory introduced in this change, all Artifacts with the same logical source root share a single ArtifactRoot instance. PiperOrigin-RevId: 192513819
* Expose "mnemonic" and "env" on skylark "Action" objects.Gravatar cparsons2018-04-11
| | | | | RELNOTES: Exposed "mnemonic" and "env" fields on skylark "Action" objects. PiperOrigin-RevId: 192479783
* Add CommandLinesAndParamFiles class.Gravatar tomlu2018-04-11
| | | | | | | | | | | This class is currently unused. In order to keep CL sizes down we introduce this class prior to using it in the spawn runners. This class can manage an action's list of command lines and param files. For instance, SpawnAction will contain one of these instances (instead of keeping a single command line object and adding param file write actions to the action graph). At spawn execution time, the spawn runners will use this class to resolve the list of command lines and param files into a master argument list + some number of param files that need to be written. The local spawn runners can simply write the param files using a helper. In the distributed cases the param files are ready-made VirtualActionInputs that can be added to the Spawn's other inputs. RELNOTES: None PiperOrigin-RevId: 192439501
* Also explicitly log the filesystem in ArtifactFactory.validatePathGravatar shahan2018-04-09
| | | | PiperOrigin-RevId: 192167264
* Serialize Artifacts by their root-relative path rather than their exec path. ↵Gravatar janakr2018-04-08
| | | | | | The exec path contains redundant information when considered with the ArtifactRoot, so, since the ArtifactRoot is memoized, we can save space and time by only serializing the non-redundant root-relative path part. PiperOrigin-RevId: 192076469
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-03 13:04:43 +0000