aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/actions
Commit message (Collapse)AuthorAge
...
* Add CommandLinesAndParamFiles class.Gravatar tomlu2018-04-11
| | | | | | | | | | | This class is currently unused. In order to keep CL sizes down we introduce this class prior to using it in the spawn runners. This class can manage an action's list of command lines and param files. For instance, SpawnAction will contain one of these instances (instead of keeping a single command line object and adding param file write actions to the action graph). At spawn execution time, the spawn runners will use this class to resolve the list of command lines and param files into a master argument list + some number of param files that need to be written. The local spawn runners can simply write the param files using a helper. In the distributed cases the param files are ready-made VirtualActionInputs that can be added to the Spawn's other inputs. RELNOTES: None PiperOrigin-RevId: 192439501
* Also explicitly log the filesystem in ArtifactFactory.validatePathGravatar shahan2018-04-09
| | | | PiperOrigin-RevId: 192167264
* Serialize Artifacts by their root-relative path rather than their exec path. ↵Gravatar janakr2018-04-08
| | | | | | The exec path contains redundant information when considered with the ArtifactRoot, so, since the ArtifactRoot is memoized, we can save space and time by only serializing the non-redundant root-relative path part. PiperOrigin-RevId: 192076469
* Adds more debug logging to ArtifactFactory validation.Gravatar shahan2018-04-06
| | | | | | Blacklists vfs for DynamicCodec PiperOrigin-RevId: 191951230
* Cleanup Artifact.getPath() call from ActionExecutionException.Gravatar shahan2018-04-05
| | | | PiperOrigin-RevId: 191801369
* Introduces a SourceArtifact type.Gravatar shahan2018-04-05
| | | | | | | SourceArtifact.getPath() is considered safe and eventually we may restrict getPath() to only SourceArtifact. PiperOrigin-RevId: 191768519
* Avoids calling getPath() in Artifact.toDetailString().Gravatar shahan2018-04-05
| | | | | | | | Also deletes obsolete Artifact.serializeToString() method. "Semantic" change: replaces the actual exec root with <execution_root> in the toString() representation. PiperOrigin-RevId: 191742669
* Tag DirectTraversal with @AutoCodec.Gravatar mjhalupka2018-04-03
| | | | PiperOrigin-RevId: 191484639
* Change profiling to only accept strings for its "description" argument. ↵Gravatar janakr2018-04-01
| | | | | | Profiling can hold onto objects for the duration of the build, and some of those objects may be temporary that should not be persisted. In particular, UnixGlob and its inner classes should not outlive loading and analysis. For the most part, care was taken in this CL to only use strings that required no additional construction, mainly to minimize garbage (retaining references to newly created strings is not as great a concern since only the strings corresponding to the slowest K tasks are retained, for some relatively small values of K). Action descriptions for actually executing actions are eagerly expanded because that work is minimal compared to the work of actually executing an action. PiperOrigin-RevId: 191251488
* ArtifactRoot correctly switches output base when serializing.Gravatar shahan2018-03-28
| | | | PiperOrigin-RevId: 190779535
* Ensure Runner name is always set.Gravatar Googler2018-03-27
| | | | | RELNOTES: None. PiperOrigin-RevId: 190617155
* Simplify Executor.getSpawnActionContext() by removing mnemonic: spawn is enough.Gravatar Googler2018-03-26
| | | | | RELNOTES: None PiperOrigin-RevId: 190544948
* Simplified ActionContextConsumer by having it operate on a new class which ↵Gravatar Googler2018-03-26
| | | | | | holds a variety of strategy/context maps. PiperOrigin-RevId: 190491357
* Bug fix to allow timeout errors for remote spawns.Gravatar Googler2018-03-20
| | | | | | TESTED=unit RELNOTES: None PiperOrigin-RevId: 189818481
* Begins cleanup to allow ActionFS to be injected into all action executions.Gravatar shahan2018-03-15
| | | | PiperOrigin-RevId: 189244665
* Splits the iterableMarshaller into runtime codecs.Gravatar shahan2018-03-14
| | | | | | Makes NestedSetCodec into a runtime codec instead of a Marshaller. PiperOrigin-RevId: 189110883
* Tags some more non-serializable classes (for serialization).Gravatar shahan2018-03-13
| | | | PiperOrigin-RevId: 188930446
* Fix broken links and formattingGravatar brandjon2018-03-10
| | | | | RELNOTES: None PiperOrigin-RevId: 188625855
* ActionOwner references only BuildEvent instead of full BuildConfigurationGravatar shahan2018-03-10
| | | | | | This reduces the size of its serialized representation. PiperOrigin-RevId: 188597127
* Expose an actions provider on RuleConfiguredTarget instances.Gravatar cparsons2018-03-06
| | | | | | | | | Given a target (for example from a skylark aspect), one will be able to access a list of actions that the target generated using "target.actions". This is without additional memory footprint. Actions themselves are not fully exposed to skylark (and thus there isn't much meaning to gather from them in skylark yet). Access methods will follow soon. RELNOTES: None. PiperOrigin-RevId: 188098079
* Automated rollback of commit 96145511b34a2d7be834e3eb05927674e875c813.Gravatar janakr2018-03-05
| | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Fixed issue that led to rollback. RELNOTES: None *** Original change description *** Automated rollback of commit eee53d3a33dde441f7e7adaecde81ef2d3db7c1b. *** Reason for rollback *** breaks Blaze_CorpTest *** Original change description *** @AutoCodec ConfiguredTargetValue. PiperOrigin-RevId: 187869560
* Automated rollback of commit eee53d3a33dde441f7e7adaecde81ef2d3db7c1b.Gravatar dannark2018-03-04
| | | | | | | | | | | | *** Reason for rollback *** breaks Blaze_CorpTest *** Original change description *** @AutoCodec ConfiguredTargetValue. PiperOrigin-RevId: 187783702
* Revamp discussion of executable/test rulesGravatar brandjon2018-03-02
| | | | | | | Rewrite that part of rules.md and update rule() docs. Also clarify the Action object's docs to put it in context. RELNOTES: None PiperOrigin-RevId: 187636896
* @AutoCodec ConfiguredTargetValue.Gravatar cpeyser2018-03-02
| | | | PiperOrigin-RevId: 187635570
* Add execution platform information to the action key.Gravatar John Cater2018-03-02
| | | | | | | | | | Needed so that actions are properly invalidated and re-run when the platform's remote execution properties change. Fixes #4756. Closes #4728. Change-Id: Ia227d110c10c9aad2164cefa8a90361cc4359332 PiperOrigin-RevId: 187621349
* Deletes CODEC fields now that they are no longer needed.Gravatar shahan2018-02-28
| | | | PiperOrigin-RevId: 187397314
* Refactor the AbstractAction computeKey method to be easier to add invariantsGravatar jcater2018-02-28
| | | | | | for every type of action. PiperOrigin-RevId: 187368369
* Deletes AutoCodec.Strategy.SINGLETON now that we have @AutoCodec field tags.Gravatar shahan2018-02-28
| | | | | | This also gets rid of some static initialization cycles which we should try very hard to avoid in the future. PiperOrigin-RevId: 187334087
* Fail gracefully on conflicting actions generated by an aspect. These can ↵Gravatar janakr2018-02-26
| | | | | | come from Skylark, so we shouldn't crash. As a safety measure, subclasses of ActionLookupValue are now responsible for detecting action conflicts themselves. PiperOrigin-RevId: 187095271
* Revamp rules.md documentation for filesGravatar brandjon2018-02-26
| | | | | | | Unified "Files" and "Output files" sections. Renamed "declared files" to "predeclared files" to avoid confusion with "declare_file()"/"declare_directory()". RELNOTES: None PiperOrigin-RevId: 187017607
* @AutoCodec FilesetOutputConfiguredTarget.Gravatar janakr2018-02-23
| | | | PiperOrigin-RevId: 186837642
* Fix Fix crash on incremental builds across configuration changes when using ↵Gravatar felly2018-02-23
| | | | | | | | | | Skyframe native Filesets which reference output files. We were failing to override equality of Artifact to use the artifact owner. See the javadocs on ArtifactSkyKey for more discussion of this. Before this change, the weak interning of keys done in LegacySkyKey and FilesetEntryKey spuriously matched keys across incremental builds in cases where Artifacts differed only in their owner. PiperOrigin-RevId: 186805663
* Adding a property name to the SpawnRunner. Most runners already had it, I ↵Gravatar olaola2018-02-22
| | | | | | | | | | just add it to the interface, and include it in the SpawnResult. This will be used to categorize/aggregate spawns executed by various runners. Also, minor refinement to the cacheHit property of the SpawnResult with remote execution. RELNOTES: None TESTED=presubmit, next cl PiperOrigin-RevId: 186637978
* Add @AutoCodec to SpawnAction. Refactor the SpawnAction API for extra ↵Gravatar cpeyser2018-02-22
| | | | | | actions so that it is non-generic and does not persist a GeneratedExtension, which is hard to serialize. PiperOrigin-RevId: 186627440
* Deletes POLYMORPHIC strategy. ObjectCodec now uses runtime type information ↵Gravatar shahan2018-02-20
| | | | | | to select a codec. PiperOrigin-RevId: 186378153
* Adds ObjectCodecRegistry to {Des|S}erializationContext.Gravatar shahan2018-02-20
| | | | | | | | | | | | * AutoCodec now delegates to the registry. * Adds getSuperclass logic for resolving a codec. * Small cleanups for classes that break the registry. TODO after this change: * Explicit CODEC definitions are no longer needed and existing ones should be cleaned up. * POLYMORPHIC is no longer be needed and should be cleaned up. PiperOrigin-RevId: 186351845
* Serialize MiddlemanAction and add strategy=Strategy.Polymorphic for Action andGravatar mjhalupka2018-02-20
| | | | | | AbstractAction. PiperOrigin-RevId: 186316435
* Improve some of the names in CommandLine.Gravatar tomlu2018-02-20
| | | | PiperOrigin-RevId: 186298405
* Provide a hacky way to get an ActionInput from BinToolsGravatar ulfjack2018-02-20
| | | | | | | | | | | | We have two places in internal code that require an ActionInput for an embedded tool. The embedded tool is currently not declared as an input to the corresponding actions, which in turn causes the remote execution to stat the file directly through SingleBuildFileCache. I'm working on removing SingleBuildFileCache, so this needs to be solved. This is the quick-and-dirty fix. A more principled fix might be to check in the relevant tools instead of embedding them into Bazel, but it's also significantly more work. PiperOrigin-RevId: 186292203
* Move CommandLine, CommandLineItem, and ParamFileInfo from ↵Gravatar tomlu2018-02-15
| | | | | | | | | lib.analysis.actions -> lib.actions. These are fundamental types that want to sit alongside types like Spawn. RELNOTES: None PiperOrigin-RevId: 185887971
* Stop storing ActionTemplate in a SkyKey: it's too heavyweight. Use the same ↵Gravatar janakr2018-02-15
| | | | | | mechanism as for normal actions, have the ActionTemplateExpansionFunction look the template up when needed. PiperOrigin-RevId: 185861672
* Add a CODEC for CppCompileAction.Gravatar cpeyser2018-02-14
| | | | PiperOrigin-RevId: 185733313
* Make ActionLookupData a SkyKey. Should save some memory.Gravatar janakr2018-02-14
| | | | PiperOrigin-RevId: 185694489
* Replaces InjectingObjectCodec with dependencies threaded through ↵Gravatar shahan2018-02-13
| | | | | | (Des|S)erializationContext. PiperOrigin-RevId: 185547740
* Add a mechanism to SpawnResult to return an in-memory outputGravatar ulfjack2018-02-12
| | | | PiperOrigin-RevId: 185370712
* Update documentation about Files.Gravatar laurentlb2018-02-12
| | | | | | | | I don't think it's worth repeating things here. Let's point to the main documentation. RELNOTES: None. PiperOrigin-RevId: 185356504
* Narrow type of "configuration" field in ActionOwner. It is only ever a ↵Gravatar janakr2018-02-09
| | | | | | BuildConfiguration. PiperOrigin-RevId: 185155423
* Replace path implementation.Gravatar tomlu2018-02-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Path and PathFragment have been replaced with String-based implementations. They are pretty similar, but each method is dissimilar enough that I did not feel sharing code was appropriate. A summary of changes: PATH ==== * Subsumes LocalPath (deleted, its tests repurposed) * Use a simple string to back Path * Path instances are no longer interned; Reference equality will no longer work * Always normalized (same as before) * Some operations will now be slower, like instance compares (which were previously just a reference check) * Multiple identical paths will now consume more memory since they are not interned PATH FRAGMENT ============= * Use a simple string to back PathFragment * No more segment arrays with interned strings * Always normalized * Remove isNormalized * Replace some isNormalizied uses with containsUpLevelReferences() to check if path fragments try to escape their scope * To check if user input is normalized, supply static methods on PathFragment to validate the string before constructing a PathFragment * Because PathFragments are always normalized, we have to replace checks for literal "." from PathFragment#getPathString to PathFragment#getSafePathString. The latter returns "." for the empty string. * The previous implementation supported efficient segment semantics (segment count, iterating over segments). This is now expensive since we do longer have a segment array. ARTIFACT ======== * Remove Path instance. It is instead dynamically constructed on request. This is necessary to avoid this CL becoming a memory regression. RELNOTES: None PiperOrigin-RevId: 185062932
* Introduce -c source_file -o output_file build variablesGravatar hlopko2018-02-08
| | | | | | | | | | | | | | Prior to this cl CompileCommandLine would (almost) unconditionally emit -c and -o flags. This cl removes this logic and relies on crosstool to emit these flags. This is another small step towards platform independent C++ rules. Memory use is not affected, since the build variables used by this cl are already exposed, this cl just forces crosstools to use it. Encore of https://github.com/bazelbuild/bazel/commit/f26e8694ae78599b3e2004e3360eaf3443fa53a6. RELNOTES: None. PiperOrigin-RevId: 184981106
* Fix Fileset incrementality bug when Fileset consumes a generated file. The ↵Gravatar felly2018-02-05
| | | | | | | | | | | | | | | | native skyframe implementation was actually quite incorrect in this case: It was adding a skyframe dependency on a FileValue for the output file. Without a transitive dependency on the source files and actions that determine the output file's state, this could never work (and explains why the incremental build would fail). Instead, we now depend on the Artifact corresponding to the output file instead. This change updates the business logic RecursiveFilesystemTraversalFunction. This approach keeps the business logic of Fileset filesystem traversal centralized in RFTF. To avoid making weird recursive Skyframe nodes in the output tree, we inline Skyframe dependencies and do direct filesystem operations over the output tree. There are now three states we can be in when looking up a file: 1. Source file: As before, make a skyframe dep on the corresponding file 2. Top-level file of an output tree: Make a dep on the corresponding Artifact 3. Recursive file under an output directory: Do direct filesystem operations. It doesn't make sense to make Skyframe nodes corresponding to these files. In the future, I think we should consider failing fast on this case. RELNOTES: None PiperOrigin-RevId: 184556044