aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/rules/cpp/CppCompileAction.java
Commit message (Collapse)AuthorAge
* ActionAnalysisMetadata exposes whether loose headers are present. ↵Gravatar shahan2018-08-12
| | | | | | TransitiveVersionTable uses futures. PiperOrigin-RevId: 208402465
* Use NestedSets to store topLevelModules and discoveredModules. Also improveGravatar Googler2018-08-11
| | | | | | | | computeTransitivelyUsedModules to return early on missing values instead of starting to create datastructures. RELNOTES: None. PiperOrigin-RevId: 208351193
* Only remember discovered modules for modules. For all other compiles, we won'tGravatar Googler2018-08-11
| | | | | | | need them later. RELNOTES: None. PiperOrigin-RevId: 208345603
* Improve implementation of filterDiscoveredHeaders. Instead of creating a hugeGravatar Googler2018-08-07
| | | | | | | | | | set of all declared headers, iterate over them and remove them from a (often much smaller) set. This should improve runtime as well as amount of garbage produced. RELNOTES: None. PiperOrigin-RevId: 207710867
* Add builtin include directories to action key for CppCompileActionGravatar hlopko2018-08-01
| | | | | | | | Since builtin include directories affect validation, they have to be part of the action key. RELNOTES: None. PiperOrigin-RevId: 206960988
* Remove the need for discoverInputsStage2(). Much like every other SkyframeGravatar Googler2018-08-01
| | | | | | | | function, discoverInputs() can be implemented to return null upon encountering missing ActionExecutionValues. RELNOTES: None. PiperOrigin-RevId: 206913969
* There is no need to do input discovery or .d file analysis for modular codegen.Gravatar Googler2018-08-01
| | | | | | | | Modular codegen simply transforms a .pcm file to a .o file containing the code of inline functions of the module's headers. RELNOTES: None PiperOrigin-RevId: 206902693
* Only call filterDiscoveredHeaders() if include scanning is in effect.Gravatar lberki2018-07-31
| | | | | | | | | | | | This is a step towards making CppCompileAction work without input discovery. This works because: - filterDiscoveredHeaders() is not necessary if we don't discover any - shouldPruneModules is always false if `shouldScanIncludes` is false (see CppCompileActionBuilder#shouldPruneModules()) RELNOTES: None. PiperOrigin-RevId: 206721143
* Disable include scanning for C++ linkstamp compile actions, which are ↵Gravatar lberki2018-07-31
| | | | | | | | | supposed to be shared. Note that CppCompileAction#discoversInputs() will still almost always return true because it's still set to true when .d file pruning is enabled. RELNOTES: None. PiperOrigin-RevId: 206716704
* Prevent unnecessary file stats when looking at whether an include is in one ofGravatar Googler2018-07-25
| | | | | | | | | the declared directories. Especially when doing validation during input discovery, the discovery's over-approximation can lead to isDeclaredIn walking the full path to the root without finding a declared include directory. RELNOTES: None. PiperOrigin-RevId: 205958078
* Remove "warn" setting for hdrs_check. This has not proven useful.Gravatar Googler2018-07-23
| | | | | | | For now, implicitly convert "warn" to "loose". RELNOTES: None. PiperOrigin-RevId: 205652060
* Implement a way to do include validation as a part of input discovery. ThisGravatar Googler2018-07-23
| | | | | | | | | | | | | | | | | makes it possible to disable .d file scanning when input discovery is used without allowing the usage of undeclared headers. The way this is implemented relies on having a sand-boxed or remote execution environment and simply removes undeclared files from discovered inputs. As a result, the compiler cannot see them and can diagnose missing headers. The input discovery itself cannot (usually) diagnose undeclared headers as it is often implemented to be an over-approximation. It needs to find all used headers, but it is allowed to find more. Diagnosing these additional headers would not be useful. RELNOTES: None. PiperOrigin-RevId: 205628312
* Ensure that gathering the info for an extra action can rely on the actionGravatar Googler2018-07-19
| | | | | | | | | being executed if the action requires input discovery. Input discovery might actually change the action's command line, which in turn can become part of the file being written. RELNOTES: None. PiperOrigin-RevId: 205207109
* Pull out a class to hold the data that is transferred from a compilation actionGravatar Googler2018-07-17
| | | | | | | to the include scanner and slightly reshuffle code. RELNOTES: None. PiperOrigin-RevId: 204906167
* Simplify (hopefully) usage of modules during input discovery. Goals:Gravatar Googler2018-07-12
| | | | | | | | | | | - Don't duplicate usedModules into additionalInputs (this shouldn't be necessary). - Use ImmutableLists instead of ImmutableSets where possible to reduce memory consumption. - Use set operations to make the code more readable. RELNOTES: None. PiperOrigin-RevId: 204268489
* Skip fingerprinting stuff into a CppCompileAction's key that can only affectGravatar Googler2018-07-11
| | | | | | | | | the result of include validation if dotd file scanning (and in turn input validation) is disabled. Fingerprinting these data structures is costly as they are large NestedSets. RELNOTES: None. PiperOrigin-RevId: 204112075
* Track additionallyPrunableIncludes separately from declaredIncludeSrcs (don'tGravatar Googler2018-07-10
| | | | | | | | | | | | | | | | | | compose a nested set encompassing both). The current prunableHeaders were used in various places, at least two of which let to a duplicate iteration through the comparatively large declaredIncludeSrcs: CppCompileAction.computeKey() and CppCompileAction.getAllowedDerivedInputs(). Also do some other minor optimizations: - CcCompileActionBuilder.buildAllInputs() now just returns a NestedSet with the same order as its parameter. As inputsForValidation is almost always empty, this makes this NestedSet pass the underlying one through. - CcCompilationContext.directModuleMaps is just a list of the module maps of direct dependencies. As such, there is no use for keeping them in a NestedSet. RELNOTES: None. PiperOrigin-RevId: 203938011
* Restructure the NestedSet usage in CcCompilationContext. Goals:Gravatar Googler2018-07-10
| | | | | | | | | | | | | | | | | | - Don't use NestedSets of NestedSets. Not sure whether this is a performance improvement or has serialization benefits, but it does make understanding the structure a lot easier. - Use the same large NestedSet for all header information (whether or not they are modular and what pregrepped headers might exist). This speeds up the include scanner calls getLegalGeneratedScannerFileMap and getModularHeaders because they now iterate over just a single NestedSet and the second evaluation of a NestedSet is much cheaper. In a subsequent change, we can likely fold all three iterations over this NestedSet into one (in getLegalGeneratedScannerFileMap, getModularHeaders and getUsedModules). Measurements show a small reduction in heap usage. RELNOTES: None. PiperOrigin-RevId: 203909308
* Don't unnecessarily create a NestedSet and remove a stale comment. We do ↵Gravatar Googler2018-07-06
| | | | | | | want to use input discovery when building modules for performance reasons. RELNOTES: None. PiperOrigin-RevId: 203463009
* Add an experimental option to disable .d-file scanning for modular builds. WhenGravatar Googler2018-07-06
| | | | | | | | | | | building with C++ modules, .d file scanning is not effective as Clang (only compiler supporting Bazel's version of C++ modules) reads all of the files passed in anyway. This is not meant to be activated yet, but for running experiments. RELNOTES: None. PiperOrigin-RevId: 203459168
* Make C++ extra action properly add module file flags for discovered C++Gravatar Googler2018-07-06
| | | | | | | modules. RELNOTES: None. PiperOrigin-RevId: 203454562
* Instead of depending on mutable inputs of upstream CppCompileAction instances,Gravatar shahan2018-06-26
| | | | | | | | | CppCompileAction.discoverInputsStage2 retrieves values of discovered modules from ActionExecutionValue. This addresses a possible a correctness issue. PiperOrigin-RevId: 202162180
* Remove support for --discard_actions_after_execution.Gravatar tomlu2018-06-18
| | | | | | | The memory savings from this flag are not worth the complexity, and it interferes with action restarting. RELNOTES: Remove support for --discard_actions_after_execution. PiperOrigin-RevId: 201077905
* Remove LIPO supportGravatar hlopko2018-06-15
| | | | | RELNOTES: Support for LIPO has been fully removed. PiperOrigin-RevId: 200724578
* Add AutoProfiler-like API to ProfilerGravatar ulfjack2018-06-11
| | | | | | - migrate all startTask/completeTask pairs to the new API PiperOrigin-RevId: 200038703
* Expose C++ action names to SkylarkGravatar hlopko2018-06-07
| | | | | | | | | | | | | | This cl adds Skylark constants allowing users to specify which C++ action they want for the feature configuration Skylark API. This is done by exposing a Skylark file at @bazel_tools//tools/cpp:action_names.bzl. Skylark api to the C++ toolchain doc: https://docs.google.com/document/d/1g91BWJITcYw_X-VxsDC0VgUn5E9g0kRBGoBSpoO41gA/edit#. Progress on #4571. RELNOTES: None. PiperOrigin-RevId: 199596778
* Simplify CppCompileAction#discoverInputs().Gravatar lberki2018-06-05
| | | | | | | | | | | | | | In the process, make it so that running an extra action attached to a CppCompileAction does not change the state of the CppCompileAction (yes, this happened: if include scanning was not done, topLevelModules would be changed) There are two changes in behavior this will: introduce 1. topLevelModules will no longer be set if include scanning is not in effect, but that's okay: it's never read except when shouldPruneModules is true, and if that is true, #discoverInputsStage2() will set it. 2. Extra actions attached to CppCompileAction will not get .pcm files on their inputs that were not used by the compiler RELNOTES: None. PiperOrigin-RevId: 199285276
* Avoid unnecessary work in validating C++ inclusions. Specifically:Gravatar Googler2018-05-29
| | | | | | | | | | | | | | - declaredIncludeSrcs don't need to be looked at as they also become part of the compilation prerequisites (see CcCompilationContext.addDeclaredIncludeSrc()) - transitiveModules can never be "incorrect" as they get computed and added by Bazel itself. - declaredIncludeDirs/warnIncludeDirs can be computed lazily. As most people aim for a warning-free build, it is rarely necessary to compute either set in practice. RELNOTES: None. PiperOrigin-RevId: 198386262
* Remove CppCompileAction.optionalSourceFileGravatar hlopko2018-05-28
| | | | | | | It's unused. RELNOTES: NONE. PiperOrigin-RevId: 198316668
* Add an option --experimental_prune_cpp_input_discovery to stop C++ inputGravatar Googler2018-05-28
| | | | | | | | discovery at the boundary of modular headers. C++ modules generally should be self-contained and a using a C++ module doesn't require any of it's transitive source files to be present. PiperOrigin-RevId: 198299936
* Split user_link_flags from legacy_link_flagsGravatar hlopko2018-05-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The difference between them is that user_link_flags will stay after we remove legacy fields from the crosstool. This is an encore of https://github.com/bazelbuild/bazel/commit/2661abb96b1fe51fb726a63eb08698564a82eb20 after mitigating the memory regression. Original cl regressed 12mb (on a big enough internal project :): objsize chg instances space KB class name ------------------------------------------------------ 24 +0 190265 4459 KB com.google.common.collect.ImmutableMapEntry 40 +0 95154 3716 KB com.google.common.collect.RegularImmutableMap 47 -2 95154 2230 KB [Ljava.util.Map$Entry; 44 -1 95154 2230 KB [Lcom.google.common.collect.ImmutableMapEntry; 24 +0 95149 2230 KB com.google.devtools.build.lib.rules.cpp.CcToolchainVariables$MapVariables 16 +0 95149 1486 KB com.google.devtools.build.lib.rules.cpp.CcToolchainVariables$StringSequence 89 +0 3188 176 KB [B 64 -8 0 -743 KB com.google.devtools.build.lib.rules.cpp.LinkCommandLine 76 +0 -1918 -1323 KB [Ljava.lang.Object; 24 +0 -95149 -2230 KB com.google.devtools.build.lib.rules.cpp.CcToolchainVariables$SingleVariables ------------------------------------------------------ The reason was that before legacy_link_flags were the only build variable patched because of thinlto in CppLinkActionBuilder, so it used optimized SingleVariables subclass. With the split, the patched variables instance had 2 variables therefore it received the full blown Variables instance. That accounted for the ~7mb. The fix was to move the patching logic to the initial link variables creation and not to create patched variables at all. Now the regression is ~4.8mb, which is perfectly expected since I introduce another variable and this is the overhead of additional hashmap entry: objsize chg instances space KB class name ------------------------------------------------------ 24 +0 190418 4462 KB com.google.common.collect.ImmutableMapEntry 44 +1 31 1487 KB [Lcom.google.common.collect.ImmutableMapEntry; 16 +0 95149 1486 KB com.google.devtools.build.lib.rules.cpp.CcToolchainVariables$StringSequence 47 +0 31 744 KB [Ljava.util.Map$Entry; 89 +0 5723 315 KB [B 24 +0 5721 134 KB java.lang.String 64 -8 0 -743 KB com.google.devtools.build.lib.rules.cpp.LinkCommandLine 76 +0 -2387 -840 KB [Ljava.lang.Object; 24 +0 -95149 -2230 KB com.google.devtools.build.lib.rules.cpp.CcToolchainVariables$SingleVariables ------------------------------------------------------ I'll pay this dept back once we get rid of legacy crosstool fields (= removal of legacy_link_flags variable). RELNOTES: None. PiperOrigin-RevId: 197574153
* Partial rollback of commit 2661abb96b1fe51fb726a63eb08698564a82eb20.Gravatar hlopko2018-05-18
| | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks perf regtest *** Original change description *** Split user_link_flags from legacy_link_flags The difference between them is that user_link_flags will stay after we remove legacy fields from the crosstool. RELNOTES: None. PiperOrigin-RevId: 197180518
* Rename CommandAction.getEnvironment, add AbstractAction.getEnvironmentGravatar ulfjack2018-05-17
| | | | | | | | | | | | | | | This is a cleanup to clarify the code. 1. The getEnvironment method in the CommandAction interface does not have access to the clientEnv, so it's return value is necessarily incomplete. Rename to getIncompleteEnvironmentForTesting. 2. Add a final getEnvironment method to AbstractAction, which returns the ActionEnvironment, which is intended to be a complete description of the intended final environment of the action (technically, of any spawn running within the action). This is not currently used, but is provided to prevent action subclasses to add such a method (it may be used in the future). PiperOrigin-RevId: 196991091
* Fix Cpp{Compile,Link}Action environment and cache key computationGravatar ulfjack2018-05-17
| | | | | | | | | | These were previously ignoring the inhertied environment, i.e., --action_env=PATH did _not_ result in the PATH variable being forwarded from the client environment. Fixes #5142. PiperOrigin-RevId: 196966822
* Split user_link_flags from legacy_link_flagsGravatar hlopko2018-05-16
| | | | | | | | The difference between them is that user_link_flags will stay after we remove legacy fields from the crosstool. RELNOTES: None. PiperOrigin-RevId: 196940832
* Extract logic for dealing with CROSSTOOL values and build variables into ↵Gravatar rosica2018-05-14
| | | | | | | separate class RELNOTES: None. PiperOrigin-RevId: 196517537
* C++: Renames CcCompilationContextInfo to CcCompilationContextGravatar plf2018-05-14
| | | | | | | Since it's not a provider, it doesn't need the Info suffix anymore. RELNOTES:none PiperOrigin-RevId: 196498526
* Adds an action-scoped filesystem.Gravatar shahan2018-05-09
| | | | PiperOrigin-RevId: 195973862
* Clean up code that directly imports nested classes like Builder, Entry, etc.Gravatar jcater2018-05-02
| | | | PiperOrigin-RevId: 195094385
* Properly report completion of shared actions with input discoveryGravatar hlopko2018-04-24
| | | | | | | | | | | | | | Currently we report "Analyzing" when include scanning runs. But since we can have shared C++ compile actions, only one of the group will be executed and only one will be reported completed. Remaining shared actions currently stay with "analyzing" forever. This cl makes sure that these actions are properly handled when finished. This is an encore of https://github.com/bazelbuild/bazel/commit/24f19ec2679dd93b1ac5b06e46f3b35807d6e217. In this incarnation I make sure that all actions that discover inputs are consistent in reporting their Analyzing status. Originally only CppCompileAction was doing that. Apparently we have more actions that discover inputs (e.g. LtoBackendAction) but these were not reporting Analyzing and therefore crashing on preconditions. This cl makes sure that all actions discovering inputs report their analyzing status. RELNOTES: None PiperOrigin-RevId: 194066513
* Accept included headers which are provided by tree artifacts.Gravatar carmi2018-04-23
| | | | | | | This allows a C++ file to include headers from a tree artifact, and pass header inclusion checks. RELNOTES: None. PiperOrigin-RevId: 193967617
* Make action_config.tools include crosstool top path by defaultGravatar hlopko2018-04-20
| | | | | | | | | | | We always prepend crosstool top path anyway, so let's create Tool instances with the already prepared path. I don't expect this affects memory usage since we don't have many cc_toolchain targets in the graph. RELNOTES: None PiperOrigin-RevId: 193660445
* Starts threading ActionExecutionContext to sites calling getPath() only withinGravatar shahan2018-04-11
| | | | | | Action execution. PiperOrigin-RevId: 192488641
* Fix Artifact.getPath() call in CppCompileAction.validateInclusionsGravatar shahan2018-04-09
| | | | PiperOrigin-RevId: 192153123
* Replaces JavaSerializableCodec with DynamicCodec as the defaultGravatar shahan2018-04-04
| | | | | | | | | * Skylark serialization was previously dropping location in error, which this fixes. * Deletes a lot of codecs with fidelity issues (DynamicCodec has full fidelity). * Deletes EnumRuntimeCodec which can now be replaced with the superior EnumCodec. * This should eventually allow us to delete Serializable from all Blaze. The remaining blocker is NoSuchPackageExceptionCodec. PiperOrigin-RevId: 191603929
* C++: Rename CcCompilationInfo to CcCompilationContextInfo.Gravatar plf2018-04-03
| | | | | | | | | This is done so that the name CcCompilationInfo can be used for the C++ provider that will wrap all providers for compilation, similar to JavaInfo in Java. RELNOTES:none PiperOrigin-RevId: 191445120
* Remove CcToolchainProvider#getEnvironment() and all the supporting ↵Gravatar lberki2018-04-03
| | | | | | | | | | | infrastructure. This was added in unknown commit to provide a different environment to Apple toolchains, then its use removed in unknown commit in favor of getting the environment variables from the CToolchain proto. I haven't done my research if that's a better approach, but it looks like it (the less hard-coded stuff we have in Java, the better), but worst of all is surely to have *two* such mechanisms. RELNOTES: None. PiperOrigin-RevId: 191411878
* Change profiling to only accept strings for its "description" argument. ↵Gravatar janakr2018-04-01
| | | | | | Profiling can hold onto objects for the duration of the build, and some of those objects may be temporary that should not be persisted. In particular, UnixGlob and its inner classes should not outlive loading and analysis. For the most part, care was taken in this CL to only use strings that required no additional construction, mainly to minimize garbage (retaining references to newly created strings is not as great a concern since only the strings corresponding to the slowest K tasks are retained, for some relatively small values of K). Action descriptions for actually executing actions are eagerly expanded because that work is minimal compared to the work of actually executing an action. PiperOrigin-RevId: 191251488
* Add a binary to CppCompileAction that is responsible for grepping header ↵Gravatar ajmichael2018-03-07
| | | | | | | files for include statments. This binary is currently only used for an internal feature - but that feature may be supported externally eventually. RELNOTES: None PiperOrigin-RevId: 188173513
* Do not upload link inputs of the linkstamp compile actions to the executorGravatar hlopko2018-03-05
| | | | | | | | | | | | | | | | | Link inputs are there only to invalidate the action, they don't affect the execution of the compile action. This cl optimizes the action execution by not having to collect unnecessary inputs into the sandbox. Since this is CppCompileAction and nothing is easy here, let me elaborate on my assumptions. The goal of this cl is to stop passing unneeded inputs into Spawns. This is implemented by introducing a dispensableInputs field to the CppCompileAction. SpawnGccStrategy then doesn't forward these further. I'm not adding dispensableInputs to the ActionKey. I think it's enough to put them to the inputs and into the updateActionInputs method. RELNOTES: None. PiperOrigin-RevId: 187849482
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-07 22:08:05 +0000