aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/analysis/actions
Commit message (Collapse)AuthorAge
* Move CommandLine, CommandLineItem, and ParamFileInfo from ↵Gravatar tomlu2018-02-15
| | | | | | | | | lib.analysis.actions -> lib.actions. These are fundamental types that want to sit alongside types like Spawn. RELNOTES: None PiperOrigin-RevId: 185887971
* Stop storing ActionTemplate in a SkyKey: it's too heavyweight. Use the same ↵Gravatar janakr2018-02-15
| | | | | | mechanism as for normal actions, have the ActionTemplateExpansionFunction look the template up when needed. PiperOrigin-RevId: 185861672
* Use AutoCodec to generate CODECs for a few more provider types that areGravatar cpeyser2018-02-15
| | | | | | exported by cc_library. PiperOrigin-RevId: 185852115
* Replace path implementation.Gravatar tomlu2018-02-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Path and PathFragment have been replaced with String-based implementations. They are pretty similar, but each method is dissimilar enough that I did not feel sharing code was appropriate. A summary of changes: PATH ==== * Subsumes LocalPath (deleted, its tests repurposed) * Use a simple string to back Path * Path instances are no longer interned; Reference equality will no longer work * Always normalized (same as before) * Some operations will now be slower, like instance compares (which were previously just a reference check) * Multiple identical paths will now consume more memory since they are not interned PATH FRAGMENT ============= * Use a simple string to back PathFragment * No more segment arrays with interned strings * Always normalized * Remove isNormalized * Replace some isNormalizied uses with containsUpLevelReferences() to check if path fragments try to escape their scope * To check if user input is normalized, supply static methods on PathFragment to validate the string before constructing a PathFragment * Because PathFragments are always normalized, we have to replace checks for literal "." from PathFragment#getPathString to PathFragment#getSafePathString. The latter returns "." for the empty string. * The previous implementation supported efficient segment semantics (segment count, iterating over segments). This is now expensive since we do longer have a segment array. ARTIFACT ======== * Remove Path instance. It is instead dynamically constructed on request. This is necessary to avoid this CL becoming a memory regression. RELNOTES: None PiperOrigin-RevId: 185062932
* Add type assertion that TreeFileArtifact's parentTreeArtifact member is aGravatar cpeyser2018-02-05
| | | | | | SpecialArtifact. PiperOrigin-RevId: 184539696
* Allow CustomCommandLine's mapFn to expand each object into multiple items.Gravatar tomlu2018-02-01
| | | | | | This is needed to migrate JavaCompileAction away from CustomMultiArgv. PiperOrigin-RevId: 184136486
* Improve safety of NestedSetFingerprintCache by detecting multiple instances ↵Gravatar tomlu2018-01-31
| | | | | | | | | | | | of the same mapFn class. This code tries to add protection against the user creating new mapFn instances per-rule. This would cause the nested set cache to be computed per-rule instead of shared across rule instances, causing memory bloat and slowdowns. Since this can only happen in native code, we can get away with detecting this and crashing blaze. I think this is a better choice than silently allowing it / falling back to slow computations. The user can override this behaviour by inheriting from CommandLineItem.CapturingMapFn, in which case the user is explicitly saying they assume responsibility for the number of instances of the mapFn the application will use. PiperOrigin-RevId: 184061642
* Use nested set cache in key computation for CustomCommandLine.Gravatar tomlu2018-01-29
| | | | | RELNOTES: None PiperOrigin-RevId: 183727976
* Add CommandLineItem interface.Gravatar tomlu2018-01-25
| | | | | | | | This interface makes it clearer in the type system exactly how items that go into a CustomCommandLine are turned into strings. It is a preparatory change to allow command line fingerprints to be more cheaply calculated, but it is valuable in itself from a code quality standpoint. PiperOrigin-RevId: 183274022
* Rename Root to ArtifactRoot.Gravatar tomlu2018-01-16
| | | | | | This is slightly more descriptive, and we will potentially want to use the name Root for a broader concept shared between ArtifactRoot and RootedPath. PiperOrigin-RevId: 182082367
* Fix Action subclasses so that the execution platform is properly set inGravatar John Cater2018-01-04
| | | | | | | | | more cases. Part of #4128. Change-Id: Ife5e4581f91ac07931d193ed5eaa256aab3ad047 PiperOrigin-RevId: 180826445
* Set the execution platform on a new SpawnAction based on the RuleContext.Gravatar John Cater2017-12-18
| | | | | | | Part of #4128. Change-Id: Ic46d2db2017b6cf4c14a91653ab75b3381b80b5a PiperOrigin-RevId: 179426362
* Allow not recognized Artifact category to be handled in Analysis phaseGravatar Googler2017-12-13
| | | | | | | Nonexistent artifact categories or categories that are not supported by the action config now throw InvalidConfigurationException instead of ExpansionException. This allows the checked exception to be caught and reported as a RuleErrorException in Analysis phase. RELNOTES: None. PiperOrigin-RevId: 178919727
* Add --experimental_java_coverage.Gravatar elenairina2017-12-05
| | | | | | | | | | | | | | | | | | Context: java_import or other custom rules (genrules or Skylark) do not propagate coverage information. Coverage metadata is retrieved from the compilation information and it is passed around through providers as Artifact(s). The problem with the current implementation is that there is no way of retrieving instrumentation metadata from arbitrary jars provided by java_import or other custom rules. --experimental_java_coverage solves the issue presented above ONLY for the java rules (has no effect for android/[]/etc). Implementation details: * For each build jar create a .txt file containing the relative path of each Java file. This file is included in the build jar. It is used for recreating the correct path for each covered file when included in the coverage report. * java_binary/java_test will set 3 environment variables: 1) JACOCO_METADATA_JAR - in experimental mode will be a txt file containing all the jars considered for collecting coverage (JacocoCoverageRunner filters out the ones that don't have .uninstrumented.class files). In non-experimental mode will be a jar containing all the instrumented class files. 2) JACOCO_MAIN_CLASS - The main class to be called for the current coverage run. Previously this information was embedded in the JACOCO_METADATA_JAR's manifest. 3) JACOCO_JAVA_RUNFILES RELNOTES: --experimental_java_coverage is available for testing. PiperOrigin-RevId: 177941471
* Added execution platform to the ActionExecutionMetadata.Gravatar John Cater2017-12-04
| | | | | | | Part of #4128. Change-Id: Id822d3ae6f8daf7c92a75bd8bd28590d4f625845 PiperOrigin-RevId: 177905460
* Remove some deadish Windows-related runfiles code.Gravatar Benjamin Peterson2017-12-04
| | | | | | | | | | | | | | | In particular, SymlinkTreeAction no longer needs to accept artifacts as an input. --experimental_enable_runfiles now immediately reports an error on Windows. This mostly unwinds e4974e4cc6aeb437d36b3b36eb20142b7120fb16 ("Separate runfiles middlemen into two layers") and 41f4456ac2348bef66739194853a1ddadcbb887e ("Make runfiles tree creation on Windows depend on the artifacts of the actual runfiles."). See https://groups.google.com/d/msg/bazel-dev/btOAgxv434g/bDhTOOePAgAJ. Change-Id: Iac3308669bfc07abfd1c91445922269d8fdc2a26 PiperOrigin-RevId: 177837504
* Add ActionKeyContext to Action#getKey.Gravatar tomlu2017-11-29
| | | | | | | This key context can be used by actions to share partial key computations, for instance when computing MD5s for nested sets. RELNOTES: None PiperOrigin-RevId: 177359607
* Return SpawnResults using a List instead of a Set.Gravatar ruperts2017-11-21
| | | | | | | | | Currently we don't care about the list order of SpawnResults. However, we may care about the list order later. Also, if the equals() method for SpawnResults is ever changed then it may be problematic to return SpawnResults in a Set. Aside: ActionResults use SpawnResults to calculate cumulative execution times for Actions, and may provide other metrics in future. RELNOTES: None. PiperOrigin-RevId: 176579460
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Reduce iterations and copies over inputs in ActionSpawn.Gravatar Jakob Buchgraber2017-10-30
| | | | | | | | | | | | | | | | The current implementation copies all inputs into a list and then iterates over them twice to remove filesets and manifests. We can do the filtering in one pass instead and avoid the copying to a list to save memory. In addition, the Action.getInputs() method often returns inputs stored in a NestedSet and we want to allow a caller to have access to it, so that he isn't forced to flatten it. A call can do that by checking if ActionSpawn.getInputFiles() is an instance of NestedSetView. Change-Id: I2fa8904827030c04bf29b14f0f3f942877c317a4 PiperOrigin-RevId: 173881256
* Add Path argument to supportsModifications, supports*LinksNatively.Gravatar tomlu2017-10-20
| | | | | | | | This works better with UnionFileSystem, as (eg.) different mapped file systems might not agree on whether they are read-only. No actual code changes have been made that actually uses this argument, so this should be a behaviour no-op. PiperOrigin-RevId: 172782759
* Actions now have the option of returning an ActionResult, containing a ↵Gravatar ruperts2017-10-18
| | | | | | | (possibly empty) set of SpawnResults created during execution of the Action. RELNOTES: None. PiperOrigin-RevId: 172529328
* More SpawnResult-related plumbing changes to Actions, Strategies, ↵Gravatar ruperts2017-10-07
| | | | | | | | | ActionContexts, etc., so that SpawnResult metadata is returned upwards. Note that the TODOs mostly refer to changes that will appear in a subsequent CL (a CL to return SpawnResults, contained in ActionResults, from Actions/AbstractActions). I split off the remaining SpawnResult-related changes into this CL and kept the ActionResult-related changes separate. RELNOTES: None. PiperOrigin-RevId: 171355611
* Windows,launcher: add LauncherFileWriteActionGravatar Laszlo Csomor2017-10-06
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Introduce LauncherFileWriteAction, a FileWriteAction that can create a native {java,sh,py}_binary launcher from the launcher stub and the user-specified launch information. The LauncherFileWriteAction class does not use the "copy" command of cmd.exe so it's not restricted with path lengths. The class stores a LaunchInfo object, which describes the launch data that the launcher stub reads to identify its payload. The LaunchInfo is a lazy object, similar to CustomCommandLine. LaunchInfo won't fully construct the launch data until it is about to write the data to the output. Thus LaunchInfo is memory efficient because it won't create any String objects in the analysis phase that it would only read in the execution phase. Fixes https://github.com/bazelbuild/bazel/issues/3802 Change-Id: I4ddd83369e7131d42e2e9b35f105ad2dc60bcc52 PiperOrigin-RevId: 171115105
* Don't symlink into the execroot if possible in SymlinkAction: instead, ↵Gravatar janakr2017-10-02
| | | | | | symlink directly to the target artifact. Also offer the option to not provide the package roots to create the execroot: we would like to avoid the execroot if possible. PiperOrigin-RevId: 170515263
* Fix various linter errorsGravatar ruperts2017-09-29
| | | | | RELNOTES: None. PiperOrigin-RevId: 170418147
* Automated rollback of commit 411039319c1c67f2b9c8a7ada9e0a11d9bd4023f.Gravatar elenairina2017-09-28
| | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks coverage for android_test (N/A). Can be reproduced with unknown commit. *** Original change description *** Rollforward change of Java coverage logic. RELNOTES: None. *** Original change description *** Automated rollback of commit 8d6fc64b18c7e35b93f5c43dae1dbd2f8cae2147. PiperOrigin-RevId: 170322801
* Add charset to action key computation of ParameterFileWriteAction.Gravatar twerth2017-09-27
| | | | | RELNOTES: None PiperOrigin-RevId: 170088757
* Ensure that SpawnResults make their way back to the SpawnActions that caused ↵Gravatar ruperts2017-09-27
| | | | | | | them to be created. RELNOTES: None. PiperOrigin-RevId: 170058295
* Rollforward change of Java coverage logic.Gravatar elenairina2017-09-26
| | | | | | | | | | RELNOTES: None. *** Original change description *** Automated rollback of commit 8d6fc64b18c7e35b93f5c43dae1dbd2f8cae2147. PiperOrigin-RevId: 170038845
* Automated rollback of commit 743dc14f9f30b80d6d821612f77186afb025477d.Gravatar dslomov2017-09-19
| | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Rollforward with a fix *** Original change description *** Automated rollback of commit 0ee3aa622fc13b8a5072ebddf5cd65823413b4ff. *** Reason for rollback *** Likely causing artifact conflicts for middleman artifacts in some cases due to accidental change of getMiddlemanDir() to getBinDir() in RunfilesSupport.createManifestMiddleman. *** Original change description *** Cleanup ActionConstructionContext. Do not expose the underlying Rule. RELNOTES: None. PiperOrigin-RevId: 169241011
* Implement support for Skylark param files.Gravatar tomlu2017-09-19
| | | | PiperOrigin-RevId: 169234249
* Automated rollback of commit 0ee3aa622fc13b8a5072ebddf5cd65823413b4ff.Gravatar ulfjack2017-09-19
| | | | | | | | | | | | | | | *** Reason for rollback *** Likely causing artifact conflicts for middleman artifacts in some cases due to accidental change of getMiddlemanDir() to getBinDir() in RunfilesSupport.createManifestMiddleman. *** Original change description *** Cleanup ActionConstructionContext. Do not expose the underlying Rule. RELNOTES: None. PiperOrigin-RevId: 169230095
* Cleanup ActionConstructionContext.Gravatar dslomov2017-09-19
| | | | | | | Do not expose the underlying Rule. RELNOTES: None. PiperOrigin-RevId: 169109552
* Change param file arg string to a format string instead of a prefix string.Gravatar tomlu2017-09-18
| | | | | | Makes it easier to do Skylark param file support which uses a format string. PiperOrigin-RevId: 169019600
* Support multiple command lines / params files in SpawnAction.Gravatar tomlu2017-09-15
| | | | | | This is necessary for the upcoming Skylark implementation of param files. PiperOrigin-RevId: 168744486
* Inline CustomCommandLine methods that can be expressed using VectorArg.Gravatar tomlu2017-09-12
| | | | | | This whole CL was done using IDE refactoring tools and should be safe. PiperOrigin-RevId: 168275575
* Remove direct argument adding methods from SpawnAction.Builder.Gravatar tomlu2017-09-05
| | | | | | These are now unused. Users are expected to add command lines directly, using (say) CustomCommandLine. PiperOrigin-RevId: 167554157
* Automated rollback of commit 8d6fc64b18c7e35b93f5c43dae1dbd2f8cae2147.Gravatar plf2017-09-04
| | | | PiperOrigin-RevId: 167480127
* Rolling forward Java coverage.Gravatar elenairina2017-09-01
| | | | | | | Instead of passing all the runtime jars in the environment variable, we now write them all to a file and store the file path in the env variable, jacoco runner reading the jars from there. Changes on Jacoco runner side are here: https://github.com/bazelbuild/bazel/commit/05418b33dd87d63e2653e594d462b2aedb0e22e5 RELNOTES: A new Java coverage implementation is available. Makes possible coverage for Skylark JVM rules. PiperOrigin-RevId: 167248966
* Restore VectorArg to CustomCommandLine's public interface.Gravatar tomlu2017-08-24
| | | | | | | | This CL intends to remove 81 overloads from CustomCommandLine, replacing them with 6 overloads that operate on VectorArg. The actual inlining isn't done in this CL to make it easier to review. This CL tries to balance reducing the API contact surface area while retaining the readability and discoverability of these methods, ensuring that they will be used over methods that eagerly convert arguments to strings. PiperOrigin-RevId: 166280990
* Allow CommandLine expansion to throw an exception.Gravatar tomlu2017-08-24
| | | | | | | | | | | | | | | | | | | This paves the way for Skylark-side compact command lines that can fail during expansion. In general, expansion happens in these scenarios: * Action execution expands the command line to execute it. This is fine since we are well equipped already to handle failing actions. * In the analysis phase we expand command lines to investigate whether we need a params file. This could be moved to the execution phase, which would have the benefit of getting params files out of the action graph and saving memory. * Key computation expands the command line. This could be fixed by allowing command lines to compute their own keys (which wouldn't try to expand the command line). This could have the benefit of being more efficient. * Extra actions expand the command line to construct the extra action proto. This could maybe be deferred to the execution phase (writing the extra action), which would also be more memory efficient. For failed key computations, we return a singleton "error" key. This means multiple actions that will fail will map to the same key. These actions will necessarily fail if executed, so we should not need to worry about these ending up in the action cache. If we do manage to make the command line compute its own keys then this will become moot in the future. RELNOTES: None PiperOrigin-RevId: 166266385
* Improve CustomCommandLine interface.Gravatar tomlu2017-08-22
| | | | | | | | | | | | | | | | In converting SpawnAction.Builder (multi-thousand line CL) users directly to CustomCommandLine I didn't like the resulting loss of readability, and the methods didn't feel very discoverable. Unless it's very convenient and readable to use CustomCommandLine, people will resort to non-memory efficient patterns by default. I'm holding that CL for this, which should offer a nicer interface. This CL removes VectorArg from the API contact surface area, instead creating 64 overloads for every valid combination of parameters. Pretty sad, but the methods dispatch straight to internal helper methods so it's mostly boilerplate to the tune of +400 LOC. Other changes: * Change ImmutableCollection -> Collection and copy the args directly into the internal args vector. Saves on collection object overhead and saves users from having to create immutable copies. * Change some names, notably add -> addAll for collection methods * Create additional missing overloads * Fix JavaDoc RELNOTES: None PiperOrigin-RevId: 165943879
* Remove CustomArgv. It is unused.Gravatar tomlu2017-08-21
| | | | PiperOrigin-RevId: 165731260
* Add more type safety to CustomCommandLine.Gravatar tomlu2017-08-16
| | | | | | | | | | | | Allowing add(Object) is too loose and can easily lead to programmer mistakes. Because of type erasure, we can't use the same overload name for (eg.) add(NestedSet<String>) and add(NestedSet<Artifact>). The API is overhauled to use the same terms everywhere, eg. "add", "addPaths", "addExecPaths". This is similar to how it used to be a few CLs ago. The API is overhauled to make sure it's consistent for all types. While tedious, the facade methods immediately dispatch to internal helpers, so implementation wise it's not too heavy. While large, this CL is almost entirely an automated refactor. PiperOrigin-RevId: 165358287
* Add @CompileTimeConstant annotations to CustomCommandLine.Gravatar tomlu2017-08-16
| | | | | | | | | This enforces certain memory-efficient patterns. For deliberate use of dynamic strings, explicitly named overloads are introduced, with javadoc that guides the programmer into making the right choice. This CL is a memory no-op on benchmarks, but it tries to prevent backslide by making sure programmers make conscious choices when they construct their command lines. RELNOTES: None PiperOrigin-RevId: 165185997
* Make AndroidTest#initializeTargetDevicesTest use lazy template args.Gravatar tomlu2017-08-14
| | | | | RELNOTES: None PiperOrigin-RevId: 165015884
* Use CustomCommandLine.Builder in SpawnAction always.Gravatar tomlu2017-08-14
| | | | | | | | This takes advantage of CustomCommandLine's ability to defer argument expansion. I can't think of any situation where this would be inferior, at it also cleans up the code a little. After the execution phase when Artifact strings will have been prodded and interned, I expect the memory difference to be less pronounced. PiperOrigin-RevId: 164996323
* Improve type safety for vector args in CustomCommandLine.Gravatar tomlu2017-08-14
| | | | | | | By using generics we can help the user ensure that they pass a map function of the right type. RELNOTES: None PiperOrigin-RevId: 164984415
* Remove unused tree artifact methods from CustomCommandLine.Gravatar tomlu2017-08-14
| | | | | | These only had usages in test code. The tests could be altered to use other methods. PiperOrigin-RevId: 164977900
generated by cgit v1.2.3 (git 2.39.1) at 2024-10-19 15:54:46 +0000