aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/actions
Commit message (Collapse)AuthorAge
* Include more information about build events' referenced local files.Gravatar tomlu2018-07-04
| | | | | | | Instead of just a path, events now include information about the type of file (output, source file, stdout/stderr, test logs, etc.). This information can be used by the uploaders to determine a) whether to upload, b) what kind of lease to give the files. RELNOTES: None PiperOrigin-RevId: 203285549
* Relax the threshold for calling getDigestInExclusiveMode().Gravatar Googler2018-07-02
| | | | | | | | | If the policy goal is to minimize disk seeks, this should be equally good. Second attempt, with test update. RELNOTES: None. PiperOrigin-RevId: 202907857
* Eliminates unnecessary comparisons in ActionInputMap.resize.Gravatar shahan2018-06-29
| | | | PiperOrigin-RevId: 202644128
* Add an interner for non source artifacts that are deserialized.Gravatar mjhalupka2018-06-27
| | | | PiperOrigin-RevId: 202311773
* Instead of depending on mutable inputs of upstream CppCompileAction instances,Gravatar shahan2018-06-26
| | | | | | | | | CppCompileAction.discoverInputsStage2 retrieves values of discovered modules from ActionExecutionValue. This addresses a possible a correctness issue. PiperOrigin-RevId: 202162180
* Add some debugging info to ArtifactFactory, and allow tests to access the ↵Gravatar janakr2018-06-22
| | | | | | ArtifactResolverSupplier in SkyframeExecutor. PiperOrigin-RevId: 201705857
* Add a method in FileArtifactValue to create a new instance given an existing ↵Gravatar felly2018-06-22
| | | | | | FileStatus. PiperOrigin-RevId: 201683773
* Automated rollback of commit 45b308a62f42c2c0bcfe79dcd4046c4025a31059.Gravatar janakr2018-06-20
| | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks Javascript compilation. There's currently no way to iterate over a depset of Artifacts and deduplicate by identical paths in Skylark. This means that actions that want to do something once per Artifact in a depset (add a flag to the command line with the path of the Artifact for instance) will have duplicate entries if there are multiple Artifacts with the same path. This is not a true automated rollback, since I tried to make this as minimal as possible, to avoid merge conflicts and keep some of the benefits of the rolled back CL. In particular, the tests that were changed in the original CL to give artifacts their correct owners did not need to be changed back, since the owners are still correct. Moreover, this effectively contains rollbacks of unknown commit and https://github.com/bazelbuild/bazel/commit/39d6f89644107a8f7c080c4f4aec8527c1a73954, but keeps test coverage from those CLs as well. Comments added to CL thread where not a clean rollback (there are plenty of files not changed at all: ActionArtifactCycleReporter is the main wart, since it can still handle SkyKeys with Artifact as the argument, but Artifact is no longer the argument to any SkyKey). RELNOTES: None *** Original change description *** Make Artifact#equals take the owner into account for derived artifacts. Derived artifacts' owners are important because they are used to determine the artifact's generating action. Source artifacts' owners are not used in this way, so I left them alone. This allows us to get rid of most uses of ArtifactSkyKey. We may be able to delete it entirely in a follow-up. PiperOrigin-RevId: 201464780
* Stop holding on to the Action and ActionKeyContext in ↵Gravatar janakr2018-06-19
| | | | | | | | ActionConflictException. We can construct the exception message eagerly, since it is going to be thrown with very high likelihood in any case. The complex objects inside it made it hard to serialize. PiperOrigin-RevId: 201130522
* Remove support for --discard_actions_after_execution.Gravatar tomlu2018-06-18
| | | | | | | The memory savings from this flag are not worth the complexity, and it interferes with action restarting. RELNOTES: Remove support for --discard_actions_after_execution. PiperOrigin-RevId: 201077905
* Refactoring: uses OutputService for ActionFileSystem injectionGravatar shahan2018-06-18
| | | | | | Moves more Action-oriented from skyframe package, which has become very unwieldy, to action package. This is needed to avoid circular dependencies caused when build-base is needed for skyframe. PiperOrigin-RevId: 200996982
* Add a mechanism for build event protocol events to upload filesGravatar ulfjack2018-06-15
| | | | | | | | This should be a no-op, mostly replacing PathConverter with BuildEventArtifactUploader, since none of the implementations perform any upload yet. PiperOrigin-RevId: 200685325
* Add functionality to make certain SkyValues unshareable, meaning they are ↵Gravatar janakr2018-06-13
| | | | | | not serialized. Tag TestCompletionValue and any ActionExecutionValue coming from a NotifyOnActionCacheHit (i.e., tests) like that. To make such values really not shared, request the ActionExecutionValue from TestCompletionFunction as opposed to the ArtifactValue (propagating the unshareable bit up seemed like too much fuss, and I have a dream of getting rid of ArtifactValue anyway). PiperOrigin-RevId: 200504358
* Pass ActionFS paths through to action-level FileOutErr for the Action's ↵Gravatar felly2018-06-13
| | | | | | | stdout/stderr. RELNOTES: None PiperOrigin-RevId: 200459354
* Unify path resolution codepaths.Gravatar felly2018-06-08
| | | | | RELNOTES: None PiperOrigin-RevId: 199880252
* Make Artifact#equals take the owner into account for derived artifacts.Gravatar janakr2018-06-08
| | | | | | | | Derived artifacts' owners are important because they are used to determine the artifact's generating action. Source artifacts' owners are not used in this way, so I left them alone. This allows us to get rid of most uses of ArtifactSkyKey. We may be able to delete it entirely in a follow-up. PiperOrigin-RevId: 199836436
* ActionFS correctly tracks symlink sources.Gravatar shahan2018-06-08
| | | | PiperOrigin-RevId: 199820207
* Use unsafe String operations when writing parameter files.Gravatar tomlu2018-06-08
| | | | | | | | | When a LATIN-1 parameter file is requested, we can take advantage of the fact that JDK9 strings are (usually) stored as LATIN-1. For UTF-8, we can still optimize for the common case where a LATIN-1 string contains only ASCII characters, as these are bit-identical between UTF-8 and LATIN-1. This would still be expected to be the vast majority of parameter file contents. RELNOTES: None PiperOrigin-RevId: 199816430
* Deletes default implemetation of MetadataProvider.getInputGravatar shahan2018-06-08
| | | | | | Small misc cleanups. PiperOrigin-RevId: 199797948
* Deletes ActionInputFileCache.Gravatar shahan2018-06-07
| | | | PiperOrigin-RevId: 199732415
* Refactoring: makes the code simpler by deleting Metadata and InputArtifactData.Gravatar shahan2018-06-06
| | | | | | (minor) ActionFS now implements MetadataProvider.getInput PiperOrigin-RevId: 199575194
* MetadataProvider now provides ActionInput lookup by exec path.Gravatar shahan2018-06-05
| | | | | | Adds a helper class, ActionInputMap to do this with minimal wrapping overhead. PiperOrigin-RevId: 199391251
* Removes most ActionInputFileCache functionality.Gravatar shahan2018-06-01
| | | | | | Actual class to be removed in a later change. PiperOrigin-RevId: 198937695
* Fix `equals()` and `hashCode()` for artifacts: artifacts of different ↵Gravatar Dmitry Lomov2018-05-30
| | | | | | | | | | | | | classes are not equal. Also validate that there are no tree and file artifacts with the same exec path. Fixes #4668. Closes #5284. Change-Id: Id97c0407a476a5bfc697b4ca7b858e3d0c0f8c75 PiperOrigin-RevId: 198540425
* When in use, always return ActionFS paths from the ActionExecutionContext. ↵Gravatar felly2018-05-24
| | | | | | | Instead,defer to ActionFS regarding the choice about delegation to top-level Filesystem. RELNOTES: None PiperOrigin-RevId: 197960416
* RemoteFileArtifactValue no longer tracks mtime.Gravatar shahan2018-05-24
| | | | PiperOrigin-RevId: 197924560
* Automated rollback of commit 724bdbfa47576c67eeec5c74d594203fe05188c7.Gravatar laszlocsomor2018-05-23
| | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Change introduced flakiness in //src/test/java/com/google/devtools/build/lib:actions_test Verified the culprit commit using: bazel test //src/test/java/com/google/devtools/build/lib:actions_test --runs_per_test=100 --notest_keep_going -t- See https://github.com/bazelbuild/bazel/issues/5242. *** Original change description *** Relax the threshold for calling getDigestInExclusiveMode(). If the policy goal is to minimize disk seeks, this should be equally good. RELNOTES: None. PiperOrigin-RevId: 197720793
* Remove some dead AbstractAction constructorsGravatar ulfjack2018-05-23
| | | | | | | | The constructors which don't accept an action environment should not be called, with the exception of subclasses that don't run any subprocesses, and therefore don't need an environment. PiperOrigin-RevId: 197706734
* Relax the threshold for calling getDigestInExclusiveMode().Gravatar Googler2018-05-22
| | | | | | | If the policy goal is to minimize disk seeks, this should be equally good. RELNOTES: None. PiperOrigin-RevId: 197611813
* Remove special handling of SpawnActionContext in Executor/ActionExecContextGravatar ulfjack2018-05-22
| | | | | | | | Instead, internally look up the correct context by mnemonic. This simplifies all the callers. We still need a little bit of special casing when constructing the action context map. PiperOrigin-RevId: 197572357
* Add a helper method to compute timeouts with a default valueGravatar ulfjack2018-05-22
| | | | PiperOrigin-RevId: 197545650
* Rename BuildEventConverters to BuildEventContextGravatar ulfjack2018-05-17
| | | | | | | This is in preparation for adding options to the class, which can be used to modify the behavior of build events posted to the BEP. PiperOrigin-RevId: 196997573
* Rename CommandAction.getEnvironment, add AbstractAction.getEnvironmentGravatar ulfjack2018-05-17
| | | | | | | | | | | | | | | This is a cleanup to clarify the code. 1. The getEnvironment method in the CommandAction interface does not have access to the clientEnv, so it's return value is necessarily incomplete. Rename to getIncompleteEnvironmentForTesting. 2. Add a final getEnvironment method to AbstractAction, which returns the ActionEnvironment, which is intended to be a complete description of the intended final environment of the action (technically, of any spawn running within the action). This is not currently used, but is provided to prevent action subclasses to add such a method (it may be used in the future). PiperOrigin-RevId: 196991091
* Fix Cpp{Compile,Link}Action environment and cache key computationGravatar ulfjack2018-05-17
| | | | | | | | | | These were previously ignoring the inhertied environment, i.e., --action_env=PATH did _not_ result in the PATH variable being forwarded from the client environment. Fixes #5142. PiperOrigin-RevId: 196966822
* Slightly refactor SpawnAction to improve env handlingGravatar ulfjack2018-05-15
| | | | | | | This is in preparation for fixing env handling as well as cache key (to use env) computations in subclasses of SpawnAction. PiperOrigin-RevId: 196626495
* ActionFS cleanupGravatar shahan2018-05-11
| | | | PiperOrigin-RevId: 196290095
* Adds an action-scoped filesystem.Gravatar shahan2018-05-09
| | | | PiperOrigin-RevId: 195973862
* Narrow synchronized section of ArtifactFactory#getArtifact to avoid ↵Gravatar janakr2018-05-07
| | | | | | contention and use a striped lock. PiperOrigin-RevId: 195717688
* Do not spawn local actions when available memory is too lowGravatar Josh Pieper2018-05-04
| | | | | | | | | | | | This effectively subsumes all existing memory estimation, as it no longer matters too much how much memory is assigned to each action. WIP: This is a hack, and if it goes in, should almost certainly be hidden behind a configuration option. Closes #4938. PiperOrigin-RevId: 195378568
* Use ConcurrentHashMap directly instead of MapMaker.Gravatar Googler2018-05-03
| | | | | RELNOTES: None. PiperOrigin-RevId: 195280015
* Use the in-memory metadata in blaze as the source of truth for Fileset mappingsGravatar kush2018-05-02
| | | | | | | instead of the manifest files. RELNOTES: None PiperOrigin-RevId: 195149880
* Migrate AbstractAction and SkylarkActionFactory to buildapiGravatar cparsons2018-05-01
| | | | | RELNOTES: None. PiperOrigin-RevId: 195011200
* Clean up code that directly imports nested classes like Builder, Entry, etc.Gravatar jcater2018-05-01
| | | | PiperOrigin-RevId: 194985157
* Migrate the skylark interfaces of Artifact and ArtifactRoot to skylarkbuildapiGravatar cparsons2018-05-01
| | | | | RELNOTES: None. PiperOrigin-RevId: 194960076
* Support ParamFileActionInput in LocalSpawnRunner.Gravatar tomlu2018-04-30
| | | | | RELNOTES: None PiperOrigin-RevId: 194798051
* Adds support for remote files.Gravatar shahan2018-04-26
| | | | PiperOrigin-RevId: 194413337
* Add execution information to standalone test results.Gravatar Benjamin Peterson2018-04-26
| | | | | | | | | | | | | | | In StandaloneTestStrategy, copy as much information as SpawnResult makes available to us through to both the TestResultData and BEP's TestResult.ExecutionInfo protos. One immediate consequence is that the UI and BEP can tell you whether a test result was cached remotely. I changed Executor.getEventHandler to return an ExtendedEventHandler because it makes this change easier to test. Closes #5081. Change-Id: I94fefdcd2e029c81085076736ad13a4bdf1bae8f PiperOrigin-RevId: 194383009
* Support deferred param files.Gravatar tomlu2018-04-25
| | | | | | | | | | | | | Design doc: https://docs.google.com/document/d/1JXqwwVHYosZOgmjN8xrfTalyhiUYJ99Qe2D0qBcqZ1c The behaviour is gated on --defer_param_files (default off) and is controlled by --min_param_file_size. This CL adds support for VirtualActionInputs to LocalSpawnRunner, and all remote runners already supports them. The sandboxed runners are not yet supported, but that can be added in a future CL. This CL does not add support for spawn runner using different param file limits. This will require refactoring of the spawn strategies and runners to be viable. RELNOTES: None PiperOrigin-RevId: 194265291
* SourceArtifacts are interned on deserialization using an ArtifactFactory. ↵Gravatar cpeyser2018-04-24
| | | | | | This should reduce memory consumption in NestedSet deserialization, which currently does not recycle Artifact instances. PiperOrigin-RevId: 194083901
* Add helpful functionality to CommandLines.Gravatar tomlu2018-04-17
| | | | | | | | | We're mostly adding convenience methods. This will be used by spawn runners and in Spawn construction, but we break it out now to lessen the size of the final CL. We also pull out a CommandLineLimits class instead of using an int. This will partially help with readability, but it also likely that we will add a second constraint (max length of a single argument). RELNOTES: None PiperOrigin-RevId: 193191097
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-04 10:41:02 +0000