aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/standalone
Commit message (Collapse)AuthorAge
* Refactor TemplateExpansionAction so that various strategies can be used to ↵Gravatar kush2018-06-29
| | | | | | | | | implement it. Also clarify the behavior of the expand_template API in the presence of multiple-substitutions. RELNOTES: None PiperOrigin-RevId: 202719656
* Inline some ActionContextProvider classes into their modulesGravatar ulfjack2018-06-11
| | | | | | | | | | | | | | | This also gets rid of some boilerplate. The ExecutionTool.addActionContext method has been around for a while, but is underused. There are still a few ActionContextProvider implementations left, which are implementing other functionality besides adding action contexts. As a side effect, this change reduces null build time with a hot server on linux by about a quarter. We were running the linux sandbox twice on every build, which takes about 70ms each (on my machine), with the total null build time around 300ms. PiperOrigin-RevId: 200045145
* Remove ActionContextConsumerGravatar ulfjack2018-06-11
| | | | | | | | Instead, add some simple APIs to ExecutorBuilder and inline all the previous subclasses into their corresponding modules. This removes a bunch of boilerplate. PiperOrigin-RevId: 200017162
* Remove the on-disk caching of Xcode locations.Gravatar jmmv2018-04-28
| | | | | | | | | | | | | | | | | | | | | | | There is no need for the cache to be on disk. Originally, there was a desire to share this cache with other tools... but this never happened. And, actually, because Bazel is in control of what it runs, it can just inject the "cached" values into those tools via flags. Instead, just store the cache in-memory. This avoids having to open and read the cache on every single action that is locally executed on a Mac. Results when building a large iOS app from a clean slate show up to a 1% wall time improvement on my Mac Pro 2013 and a reduction in the variance of the measurements. This change also gets rid of the OS check from the action execution's critical path. There is not much use in checking this: if we instantiate this by mistake, the actual calls will fail. But sometimes we want to actually run this code on non-macOS systems (e.g. for unit-testing with mocked tools), so we should allow that. And this change also ensures that XcodeLocalEnvProviderTest builds and runs... RELNOTES: None. PiperOrigin-RevId: 194681802
* Simplified ActionContextConsumer by having it operate on a new class which ↵Gravatar Googler2018-03-26
| | | | | | holds a variety of strategy/context maps. PiperOrigin-RevId: 190491357
* Begins cleanup to allow ActionFS to be injected into all action executions.Gravatar shahan2018-03-15
| | | | PiperOrigin-RevId: 189244665
* Add a binary to CppCompileAction that is responsible for grepping header ↵Gravatar ajmichael2018-03-07
| | | | | | | files for include statments. This binary is currently only used for an internal feature - but that feature may be supported externally eventually. RELNOTES: None PiperOrigin-RevId: 188173513
* Replace instances of XCode with Xcode which is the proper casing for the name.Gravatar Sergio Campama2018-03-06
| | | | | | Closes #4640. PiperOrigin-RevId: 188022228
* Inject the product name in the constructor where neededGravatar ulfjack2018-02-20
| | | | | | ...instead of passing it everywhere so we can rewrite the local env. PiperOrigin-RevId: 186316859
* Split the include scanning dummy classes into their own parent classGravatar ulfjack2018-02-09
| | | | PiperOrigin-RevId: 185140903
* Split CppIncludeScanningContext out of CppCompileActionContextGravatar ulfjack2018-02-08
| | | | | | Rename IncludeScanningContext to CppIncludeExtractionContext, which more closely matches its semantics. PiperOrigin-RevId: 184974810
* Move BazelRuleClassProvider.TESTING_SUPPORT to its own class in rules/test.Gravatar lpino2018-01-29
| | | | PiperOrigin-RevId: 183672444
* actions,temp: respect TMPDIR envvarGravatar Laszlo Csomor2018-01-15
| | | | | | | Fixes https://github.com/bazelbuild/bazel/issues/4376 Change-Id: Id78bb0930044626304e54f07735db4d4b2c84720 PiperOrigin-RevId: 181959528
* Plumb exec root through to all spawn runners.Gravatar tomlu2018-01-11
| | | | | | They need this to parse input manifests. Previously we would grab the exec root from the Root, but wish to unsupport this. PiperOrigin-RevId: 181669143
* Extract LocalEnvProvider members to classes.Gravatar Laszlo Csomor2018-01-04
| | | | | | | | | | | | | | | | | | Create a PosixLocalEnvProvider and WindowsLocalEnvProvider class, with singleton instances for now. This refactoring should not change functionality, it's just a requirement for an upcoming change. That upcoming change is for these classes to respect the client environment's TMPDIR or TMP/TEMP envvars. See https://github.com/bazelbuild/bazel/issues/4376 Change-Id: I032bb6f18adf8af9e43e6bc543c09c58adae3863 PiperOrigin-RevId: 180799936
* Add a CommandLineBuilder for the process-wrapper embedded tool, and use it ↵Gravatar ruperts2017-11-30
| | | | | | | | | everywhere instead of duplicating process-wrapper --shell_arguments in Blaze. To avoid a cyclic dependency, I broke up exec/local:local into exec/local:local and exec/local:options. RELNOTES: None. PiperOrigin-RevId: 177419268
* Actions now have a temp envvar.Gravatar László Csomor2017-10-16
| | | | | | | | | | | | | | | | | | | | | Every build and test action that creates a Spawn will now have platform-specific environment variables for temp directories: - on Windows: TMP and TEMP - on Linux/Darwin: TMPDIR This is particularly important on Windows where e.g. Java programs cannot create temp directories unless there's a valid TMP or TEMP environment variable set. Fixes: - https://github.com/bazelbuild/bazel/issues/1590 - https://github.com/bazelbuild/bazel/issues/2349 - https://github.com/bazelbuild/bazel/issues/2870 Change-Id: Ib758307daf6b3a51b0f71ae5e65e5bb564dad643 PiperOrigin-RevId: 172326371
* More BUILD file refactorings.Gravatar philwo2017-09-06
| | | | | | | | | Split collect, concurrent, vfs, windows into package-level BUILD files. Move clock classes out of "util", into their own Java package. Move CompactHashSet into its own Java package to break a dependency cycle. Give nestedset and inmemoryfs their own package-level BUILD files. PiperOrigin-RevId: 167702127
* Split "shell", "graph" into their own packages.Gravatar philwo2017-08-29
| | | | PiperOrigin-RevId: 166849610
* AbstractSpawnStrategy: use ActionExecutionContext.getVerboseFailuresGravatar ulfjack2017-08-11
| | | | | | | ...instead of injecting it through the constructor. Simplify all the callers accordingly. PiperOrigin-RevId: 164955391
* Move core test classes to lib.analysis.testGravatar ulfjack2017-08-11
| | | | | | | | These are depended upon by analysis code, so need to live in the same library as lib.analysis. Moving them here makes it possible to split the build-base library into separate libraries for analysis, execution, and rules. PiperOrigin-RevId: 164847161
* Extract a common AbstractSpawnStrategy parent classGravatar ulfjack2017-07-24
| | | | | | This removes a bunch of code duplication that I previously introduced. PiperOrigin-RevId: 162909430
* Add ActionInputPrefetcher to ActionExecutionContextGravatar ulfjack2017-07-24
| | | | | | | This is more consistent with other values, and removes the need to inject it into the constructor of the various strategy implementations. PiperOrigin-RevId: 162729187
* Move ActionInputPrefetcher to the actions packageGravatar ulfjack2017-07-21
| | | | | | The plan is to add it to ActionExecutionContext, which is also there. PiperOrigin-RevId: 162656835
* Extend the SpawnRunner APIGravatar ulfjack2017-07-17
| | | | | | | | | | | | | - add an id for logging; this allows us to correlate log entries for the same spawn from multiple spawn runner implementations in the future - add a prefetch method to the SpawnExecutionPolicy; better than relying on the ActionInputPrefetcher being injected in the constructor - add a name parameter to the report method; this is in preparation for a single unified SpawnStrategy implementation - it's basically the last bit of difference between SandboxStrategy and RemoteSpawnStrategy; they're otherwise equivalent (if not identical) PiperOrigin-RevId: 162194684
* Simplify exception handling in spawn strategiesGravatar ulfjack2017-07-17
| | | | | | | | | | | | | | | | | | | | | | | | | The main change here is to only catch SpawnExecException in StandaloneTestStrategy, so all other exceptions simplify propagate up. As a result, Bazel no longer retries tests that fail with an exception, we only retry tests that actually ran, had a spawn result, and resulted in a UserExecException. That is probably what we want. Also do some cleanup: - Remove ExecException.timedOut; nobody was calling it (but there's still SpawnExecException.timedOut) - Remove SpawnActionContext.shouldPropagateExecException; all exceptions (except SpawnExecException) are now propagated by default - Remote the SandboxOptions from the SandboxStrategies; all sandboxing options are now handled by the underlying SpawnRunner implementations I'll send a followup CL to remove the UserExecException and EnvironmentalExecException types; the types don't do anything special, and there are no catch blocks in production code that catch one of these more specific types. This should fix #3322 by removing a bunch of special handling. PiperOrigin-RevId: 161960919
* Rewrite all the sandbox strategy implementationsGravatar ulfjack2017-07-12
| | | | | | | | | | | | | | - Make use of existing abstractions like SpawnRunner and SpawnExecutionPolicy. - Instead of having the *Strategy create a *Runner, and then call back into SandboxStrategy, create a single SandboxContainer which contains the full command line, environment, and everything needed to create and delete the sandbox directory. - Do all the work in SandboxStrategy, including creation and deletion of the sandbox directory. - Use SpawnResult instead of throwing, catching, and rethrowing. - Simplify the control flow a bit. PiperOrigin-RevId: 161644979
* Extract ActionContext to a top-level classGravatar ulfjack2017-06-19
| | | | PiperOrigin-RevId: 159423459
* Rewrite the Executor/ActionExecutionContext splitGravatar ulfjack2017-06-19
| | | | | | | Move everything to ActionExecutionContext, and drop Executor whereever possible. This clarifies the API, makes it simpler to test, and simplifies the code. PiperOrigin-RevId: 159414816
* Rewrite StandaloneSpawnStrategy to use LocalSpawnRunnerGravatar ulfjack2017-06-19
| | | | PiperOrigin-RevId: 159221067
* Extract the MacOS/XCode env rewrite logic into lib.exec.appleGravatar ulfjack2017-06-13
| | | | | | | | | | | | | | | | | | | Also add an interface to allow injecting that logic into LocalSpawnRunner; this is in preparation for rewriting StandaloneSpawnStrategy to use LocalSpawnRunner. At the same time, this reduces the dependencies from exec / standalone to rules.apple, which is a prerequisite for micro-Bazel. There's a small semantic change hidden here - we now only set the new XCodeLocalEnvProvider if we're actually running on Darwin, so we no longer fail execution on non-Darwin platforms if XCODE_VERSION_OVERRIDE or APPLE_SDK_VERSION_OVERRIDE is set. As a result, I moved the corresponding test from StandaloneSpawnStrategyTest to the new XCodeLocalEnvProviderTest. While I'm at it, also open source DottedVersionTest and CacheManagerTest. PiperOrigin-RevId: 158829077
* Track the test environment in Skyframe, like the action environmentGravatar ulfjack2017-06-09
| | | | | | | | Instead of passing a client env into the test strategies, use the same mechansim as --action_env, by depending on the right set of Skyframe nodes that correspond to client env entries. PiperOrigin-RevId: 158401670
* Drop CppLinkActionContextGravatar Ulf Adams2017-03-06
| | | | | | | | | | Merge the functionality into CppLinkAction, which now simply creates and runs a Spawn. Set PWD to /proc/self/cwd for Linux, and set execution info if the action needs to run on MacOS. -- PiperOrigin-RevId: 149285420 MOS_MIGRATED_REVID=149285420
* Remove all the action resource estimation codeGravatar Ulf Adams2017-03-06
| | | | | | -- PiperOrigin-RevId: 149110466 MOS_MIGRATED_REVID=149110466
* Rationalize local resource acquisitionGravatar Ulf Adams2017-03-03
| | | | | | | | | | | | | | | | | | | | | | Move all local resource acquisition to where local execution actually happens. Don't attempt to acquire resources per action, but only for individual spawns. This significantly simplifies the code. The downside is that we don't account for action-level work anymore. In general, actions should not perform any process execution themselves, but always delegate such work to a SpawnStrategy implementation. This change makes sure that every Spawn has local resources set in a way that is consistent with the previous state. However, there are two actions - Fileset and FileWrite -, which are not spawns, and so we now don't limit their concurrent execution anymore. For Fileset, all work is done in a custom Fileset-specific thread pool, so this shouldn't be a problem. I'm not sure about FileWriteAction. -- PiperOrigin-RevId: 149012600 MOS_MIGRATED_REVID=149012600
* Move UnixFileSystem to lib.unix, WindowsFileSystem to lib.windowsGravatar Ulf Adams2017-02-28
| | | | | | -- PiperOrigin-RevId: 148749485 MOS_MIGRATED_REVID=148749485
* Simplify the Spawn interface, update Executor interfaceGravatar Ulf Adams2017-02-01
| | | | | | | | | | | | | | | | | | | | Drop the Spawn.getOwner() method, which was duplicating functionality already available through Spawn.getResourceOwner(), in favor of the latter. In order to do that, change Executor.reportSubcommand to take a Spawn instance instead, which in turn requires updating all call sites. This is part of establishing Spawn as an abstraction for lower-level local and remote execution in all cases instead of passing args, env, input/output files and possibly other metadata as individual method parameters to the underlying implementations. In order for that to be maintainable, Spawn should be light-weight and easy to construct correctly, and not contain too much unnecessary data. -- PiperOrigin-RevId: 146224914 MOS_MIGRATED_REVID=146224914
* Simplify the Spawn interfaceGravatar Ulf Adams2017-02-01
| | | | | | | | | | | | | | | | | | | | | | | | Remove getSpawnInfo and asShellCommand, which are never really overridden in a useful way. asShellCommand moves to the Spawns class, and getSpawnInfo is only ever called by SpawnAction, and the implementation moves there. I'm considering using Spawn as the general lower-level abstraction for both local and remote execution. It sort of is that already, except it's not used consistently - we often pass a tuple of (args, env) plus possibly input and output files through parameter-heavy method call hierarchies instead of using this existing abstraction. However, I'm concerned about the amount of baggage it's carrying as well as the number of implementations for what is supposed to be a simple interface (or possibly even a simple value class), and this is an attempt to slim it down a bit. This should have no visible effects on builds. -- PiperOrigin-RevId: 146109838 MOS_MIGRATED_REVID=146109838
* Move some classes to lib.exec in preparation for layering enforcementGravatar Ulf Adams2016-12-21
| | | | | | | | | | | | This change is part of the mu-bazel effort, which aims to build a minimally useful Bazel binary with most extraneous functionality removed. As part of that, we want to enforce layering of packages. In particular, lib.actions must not depend on lib.rules or lib.exec. lib.rules must not depend on lib.exec. Moving these classes is a necessary step to enforce that layering. -- PiperOrigin-RevId: 142668172 MOS_MIGRATED_REVID=142668172
* Continued TestStrategy refactoring.Gravatar Ulf Adams2016-12-16
| | | | | | | | | More renaming and some reformatting to make StandaloneTestStrategy more closely resemble the internal implementation of TestStrategy. -- PiperOrigin-RevId: 142254302 MOS_MIGRATED_REVID=142254302
* Move TestStrategy to lib.exec package.Gravatar Ulf Adams2016-12-15
| | | | | | | | | | | | | | | | This is part of refactoring test strategy to unify its implementation between Bazel and Blaze (Google's internal version of Bazel), which should fix several issues in Bazel. It's also necessary to untangle lib.rules and lib.exec to enforce proper layering by separating compilation of these packages, and to provide a minimal Bazel binary. In particular, no core part of Bazel should depend on any of the rules, to facilitate moving them out of Bazel / reimplementing them in Skylark (except for some core rules like test_suite, alias, and genquery). -- PiperOrigin-RevId: 142151901 MOS_MIGRATED_REVID=142151901
* Refactor the module API to use the builder pattern for executor creation.Gravatar Ulf Adams2016-11-02
| | | | | | | This significantly simplifies several of our modules. -- MOS_MIGRATED_REVID=137713119
* Rollback of commit e6191f7209ebf3f67480bc7aa9db5ac58beb2156.Gravatar Damien Martin-Guillerez2016-10-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Roll-forward with fix. Tested with `bazel build src:srcs //src/test/...` using a bootstrapped bazel. Fixes #1923. RELNOTES[INC]: Non test-only targets can no longer depends on test-only targets. *** Original change description *** Automated [] rollback of commit a9f20b0d6459d395444c45cf5e94a899f3443633. *** Reason for rollback *** Broke Bazel CI: ERROR: /home/ci/workspace/Bazel/JAVA_VERSION/1.8/PLATFORM_NAME/linux-x86_64/src/BUILD:284:1: in filegroup rule //src:srcs: non-test target '//src:srcs' depends on testonly target '//src/java_tools/junitrunner/java/com/google/testing/junit/junit4:srcs' and doesn't have testonly attribute set. ERROR: Analysis of target '//scripts/packages:packages' failed; build aborted. http://ci.bazel.io/view/Bazel%20b... *** -- MOS_MIGRATED_REVID=136589937
* Rollback of commit f695c54a8f5f6be7adb678d305005f78e4b44424.Gravatar Philipp Wollermann2016-10-13
| | | | | -- MOS_MIGRATED_REVID=136011723
* Automated [] rollback of commit e898023ffc6c47a27312c4d3659dbeeccdb3cd37.Gravatar Yue Gan2016-10-12
| | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Make darwin sandboxing default again. *** Original change description *** Fix #1849: Sandboxing on OS X should be turned off by default for 0.3.2. This restructures the way we set the default Spawn strategy so that each BlazeModule supplying a SpawnActionContext has an ActionContextConsumer that sets its own SpawnActionContext as the default, with the BazelRulesModule being put as the last module loaded in BazelMain, so that it can override that decision - it only does, if the user explicitly specifies a --spawn_strategy flag. IMHO this is a much saner approach than... *** ROLLBACK_OF=134770427 RELNOTES: Darwin sandboxing is default. -- MOS_MIGRATED_REVID=135905657
* Fix #1849: Sandboxing on OS X should be turned off by default for 0.3.2.Gravatar Philipp Wollermann2016-09-30
| | | | | | | | | | | | This restructures the way we set the default Spawn strategy so that each BlazeModule supplying a SpawnActionContext has an ActionContextConsumer that sets its own SpawnActionContext as the default, with the BazelRulesModule being put as the last module loaded in BazelMain, so that it can override that decision - it only does, if the user explicitly specifies a --spawn_strategy flag. IMHO this is a much saner approach than the older one. So the flow is essentially this: - StandaloneActionContextConsumer sets the default strategy to "standalone". - SandboxActionContextConsumer sets the default strategy to "sandboxed", but only on Linux - BazelRulesModule sets the default strategy to the value of the --spawn_strategy flag, if it is set. -- MOS_MIGRATED_REVID=134770427
* Refactoring getTimeoutSeconds of a Spawn into Spaws.java, becaused it is usedGravatar Googler2016-09-08
| | | | | | | in places other than the sandbox code. -- MOS_MIGRATED_REVID=132436150
* Refactor our sandboxing code.Gravatar Philipp Wollermann2016-08-31
| | | | | -- MOS_MIGRATED_REVID=131817068
* modify sandbox script to allow defaultGravatar Yue Gan2016-08-29
| | | | | | | -- Change-Id: I48611500044e05c177d3a044d5d335c4a98135f4 Reviewed-on: https://bazel-review.googlesource.com/#/c/5530 MOS_MIGRATED_REVID=131593454
* Implement timeouts on Windows.Gravatar Lukacs Berki2016-08-22
| | | | | | | Makes #1664 much less acute. -- MOS_MIGRATED_REVID=130750731
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-13 22:22:04 +0000