aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/exec/local
Commit message (Collapse)AuthorAge
* Rename SpawnExecutionPolicy -> SpawnExecutionContext.Gravatar tomlu2018-04-19
| | | | | | | | This class will be used to tie a Spawn to a SpawnRunner, and isn't really a policy object. It will carry state such as the expanded inputs and expanded command line. Currently a context can be passed between different SpawnRunners. This will be addressed independently, so a context is tied to a particular spawn runner. PiperOrigin-RevId: 193501918
* Remove categories from Bazel options.Gravatar ccalvarin2018-03-28
| | | | | | | | | These have all had a chance to be categorized with the OptionDocumentationCategory enum, and the help output already uses the enum-grouped format. The "incompatible changes" category has meaning for --all_incompatible_changes and will be removed separately. RELNOTES: None. PiperOrigin-RevId: 190773778
* Refactor and cleanup the sandboxing code.Gravatar Philipp Wollermann2018-03-23
| | | | | | | | | | | | | - Remove Optional<> where it's not needed. It's nice for return values, but IMHO it was overused in this code (e.g. Optional<List<X>> is an anti-pattern, as the list itself can already signal that it is empty). - Use Bazel's own Path class when dealing with paths, not String or java.io.File. - Move LinuxSandboxUtil into the "sandbox" package. - Remove dead code and unused fields. - Migrate deprecated VFS method calls to their replacements. - Fix a bug in ExecutionStatistics where a FileInputStream was not closed. Closes #4868. PiperOrigin-RevId: 190217476
* Adding a property name to the SpawnRunner. Most runners already had it, I ↵Gravatar olaola2018-02-22
| | | | | | | | | | just add it to the interface, and include it in the SpawnResult. This will be used to categorize/aggregate spawns executed by various runners. Also, minor refinement to the cacheHit property of the SpawnResult with remote execution. RELNOTES: None TESTED=presubmit, next cl PiperOrigin-RevId: 186637978
* Inject the product name in the constructor where neededGravatar ulfjack2018-02-20
| | | | | | ...instead of passing it everywhere so we can rewrite the local env. PiperOrigin-RevId: 186316859
* Automated rollback of commit 04757dba0174d22c0a695a7ed5fe511fd13df008.Gravatar laszlocsomor2018-02-16
| | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** There's already a --test_tmpdir flag, and Java tests don't pick up this new one. More info: https://github.com/bazelbuild/bazel/issues/4621#issuecomment-366217321 *** Original change description *** tmpdir,local-exec: implement --local_tmp_root Add new flag called `--local_tmp_root`, which (if specified) tells Bazel what temp directory should locally executed actions use. Fixes https://github.com/bazelbuild/bazel/issues/4621 Related to https://github.com/bazelbuild/bazel/issues/3215 RELNOTES[NEW]: The new "--local_tmp_root=<path>" flag allows specifying the temp directory for locally executed actions. Change-Id: Ice69a5e63d0bf4d3b5c9ef4dbdd1ed1c5025f85e PiperOrigin-RevId: 185982705
* tmpdir,local-exec: implement --local_tmp_rootGravatar Laszlo Csomor2018-02-13
| | | | | | | | | | | | | | Add new flag called `--local_tmp_root`, which (if specified) tells Bazel what temp directory should locally executed actions use. Fixes https://github.com/bazelbuild/bazel/issues/4621 Related to https://github.com/bazelbuild/bazel/issues/3215 RELNOTES[NEW]: The new "--local_tmp_root=<path>" flag allows specifying the temp directory for locally executed actions. Change-Id: Ice69a5e63d0bf4d3b5c9ef4dbdd1ed1c5025f85e PiperOrigin-RevId: 185509555
* actions,temp: respect TMPDIR envvarGravatar Laszlo Csomor2018-01-15
| | | | | | | Fixes https://github.com/bazelbuild/bazel/issues/4376 Change-Id: Id78bb0930044626304e54f07735db4d4b2c84720 PiperOrigin-RevId: 181959528
* Extract LocalEnvProvider members to classes.Gravatar Laszlo Csomor2018-01-04
| | | | | | | | | | | | | | | | | | Create a PosixLocalEnvProvider and WindowsLocalEnvProvider class, with singleton instances for now. This refactoring should not change functionality, it's just a requirement for an upcoming change. That upcoming change is for these classes to respect the client environment's TMPDIR or TMP/TEMP envvars. See https://github.com/bazelbuild/bazel/issues/4376 Change-Id: I032bb6f18adf8af9e43e6bc543c09c58adae3863 PiperOrigin-RevId: 180799936
* Enable local action execution statistics collection for sandboxed actions ↵Gravatar ruperts2017-12-22
| | | | | | | | | | | | | that use either the LinuxSandboxedSpawnRunner or the ProcessWrapperSandboxedSpawnRunner. In particular, record metrics for user and system CPU execution time, block I/O and involuntary context switches. This feature is guarded behind a new option, --experimental_collect_local_sandbox_action_metrics. Note: We still need to enable execution statistics for the DarwinSandboxedSpawnRunner in a later change. RELNOTES: None. PiperOrigin-RevId: 179976217
* Move compulsory ProcessWrapperUtil parameters into its builder() method, to ↵Gravatar ruperts2017-12-20
| | | | | | | be consistent with LinuxSandboxUtil. RELNOTES: None. PiperOrigin-RevId: 179758847
* Use java.nio.file.Files.createTempDirectory() to create temporary directory ↵Gravatar ruperts2017-12-13
| | | | | | | | | | | | | | | | | in LocalSpawnRunner, to avoid a race condition. Aside: The old, real temporary directory paths looked like this: TMPDIR=/.../tmp15e_5dd5a8e8347813f5 The new, real temporary directory paths now look like this: TMPDIR=/.../local-spawn-runner.3217503035718074040 RELNOTES: None. PiperOrigin-RevId: 178903361
* Enable local action execution statistics collection when the ↵Gravatar ruperts2017-12-12
| | | | | | | | | | | LocalSpawnRunner uses the process-wrapper to run commands. In particular, record metrics for user and system CPU execution time, block I/O and involuntary context switches. This feature is guarded behind a new option, --experimental_collect_local_action_metrics. RELNOTES: None. PiperOrigin-RevId: 178856077
* If a temporary directory name clashes in the LocalSpawnRunner, try ↵Gravatar ruperts2017-12-06
| | | | | | | generating another temporary directory name instead of throwing an exception. RELNOTES: None. PiperOrigin-RevId: 178190769
* Add a CommandLineBuilder for the process-wrapper embedded tool, and use it ↵Gravatar ruperts2017-11-30
| | | | | | | | | everywhere instead of duplicating process-wrapper --shell_arguments in Blaze. To avoid a cyclic dependency, I broke up exec/local:local into exec/local:local and exec/local:options. RELNOTES: None. PiperOrigin-RevId: 177419268
* Add user and system time to CommandResults, and plumb them into SpawnResults.Gravatar ruperts2017-11-29
| | | | | RELNOTES: None PiperOrigin-RevId: 177290508
* Simplify SpawnRunner interfaceGravatar ulfjack2017-11-28
| | | | | | | | | | | | | | It turns out that the SUCCESS status is often misunderstood to mean "zero exit", even though this is clearly documented. I've decided to add another status for non-zero exit, and use success only for zero exit to avoid this pitfall. Also, many of the status codes are set, but never used. I decided to reduce the number of status codes to only those that are actually relevant, which simplifies further processing. Instead, we should add a string message for the error case when we need one - we're not using it right now, so I decided not to add that yet. PiperOrigin-RevId: 177129441
* Add a random number to action temp dirGravatar Changming Sun2017-11-20
| | | | | | | | | | Fix for #4035 @laszlocsomor Closes #4110. PiperOrigin-RevId: 176346381
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Add optional user execution time and system execution time fields to ↵Gravatar ruperts2017-11-06
| | | | | | | TerminationStatus, and also add a TerminationStatus.Builder and tests. RELNOTES: None. PiperOrigin-RevId: 174557303
* Use Durations to store wall, user and system execution time in SpawnResults, ↵Gravatar ruperts2017-11-06
| | | | | | | and make cumulative execution times available in ActionResults. RELNOTES: None PiperOrigin-RevId: 174553272
* Actions now have a temp envvar.Gravatar László Csomor2017-10-16
| | | | | | | | | | | | | | | | | | | | | Every build and test action that creates a Spawn will now have platform-specific environment variables for temp directories: - on Windows: TMP and TEMP - on Linux/Darwin: TMPDIR This is particularly important on Windows where e.g. Java programs cannot create temp directories unless there's a valid TMP or TEMP environment variable set. Fixes: - https://github.com/bazelbuild/bazel/issues/1590 - https://github.com/bazelbuild/bazel/issues/2349 - https://github.com/bazelbuild/bazel/issues/2870 Change-Id: Ib758307daf6b3a51b0f71ae5e65e5bb564dad643 PiperOrigin-RevId: 172326371
* Move SpawnResult from build.lib.exec into build.lib.actions so that e.g. ↵Gravatar ruperts2017-09-22
| | | | | | | build.lib.actions.SpawnActionContext can import SpawnResult without creating a cyclic dependency. RELNOTES: None. PiperOrigin-RevId: 169642267
* Only compute hostname once per server lifetimeGravatar hlopko2017-09-14
| | | | | | | | | | | | NetUtil.getShortHostName can take seconds on mac and on windows (like, 20!), since it performs reverse dns lookup. We already cached hostname for the BazelWorkspaceStatusModule, let's cache it for entire bazel server. Also make sure that users of the method understand it's cached. Fixes #3586. RELNOTES: None. PiperOrigin-RevId: 168691615
* More BUILD file refactorings.Gravatar philwo2017-09-06
| | | | | | | | | Split collect, concurrent, vfs, windows into package-level BUILD files. Move clock classes out of "util", into their own Java package. Move CompactHashSet into its own Java package to break a dependency cycle. Give nestedset and inmemoryfs their own package-level BUILD files. PiperOrigin-RevId: 167702127
* Split "shell", "graph" into their own packages.Gravatar philwo2017-08-29
| | | | PiperOrigin-RevId: 166849610
* Unify input prefetchingGravatar ulfjack2017-08-10
| | | | | | | All prefetching now goes through AbstractSpawnStrategy's implementation of SpawnExecutionPolicy. Make sure the sandbox runners also do this consistently. PiperOrigin-RevId: 164836877
* Rewrite the Command APIGravatar ulfjack2017-08-10
| | | | | | | | | | | | | | | | | | | | | | | | Important: the simplified API now defaults to forwarding interrupts to subprocesses. I did audit all the call sites, and I think this is a safe change to make. - Properly support timeouts with all implementations - Simplify the API - only provide two flavours of blocking calls, which require no input and forward interrupts; this is the most common usage - provide a number of async calls, which optionally takes input, and a flag whether to forward interrupts - only support input streams, no byte arrays or other 'convenience features' that are rarely needed and unnecessarily increase the surface area - use java.time.Duration to specify timeout; for consistency, interpret a timeout of <= 0 as no timeout (i.e., including rather than excluding 0) - KillableObserver and subclasses are no longer part of the public API, but still used to implement timeouts if the Subprocess.Factory does not support them - Update the documentation for Command - Update all callers; most callers now use the simplified API PiperOrigin-RevId: 164716782
* Use java.time.Duration for timeoutsGravatar ulfjack2017-08-09
| | | | PiperOrigin-RevId: 164577062
* Make the @Option annotation depend on the java version of the tagging enums.Gravatar ccalvarin2017-07-18
| | | | | | | The option filters proto dependency can be removed from the OptionsParser. This is in response to option parser users that want to avoid the bazel-internal proto file in their dependencies. RELNOTES: None. PiperOrigin-RevId: 162249778
* Extend the SpawnRunner APIGravatar ulfjack2017-07-17
| | | | | | | | | | | | | - add an id for logging; this allows us to correlate log entries for the same spawn from multiple spawn runner implementations in the future - add a prefetch method to the SpawnExecutionPolicy; better than relying on the ActionInputPrefetcher being injected in the constructor - add a name parameter to the report method; this is in preparation for a single unified SpawnStrategy implementation - it's basically the last bit of difference between SandboxStrategy and RemoteSpawnStrategy; they're otherwise equivalent (if not identical) PiperOrigin-RevId: 162194684
* LocalSpawnRunner now always logs as itselfGravatar ulfjack2017-07-14
| | | | PiperOrigin-RevId: 161944209
* Fold OptionUsageRestrictions into OptionDocumentationCategory and ↵Gravatar ccalvarin2017-07-11
| | | | | | | | | | OptionMetadataTags. These are similar, no need to have both fields. Removing the "DOCUMENTED" default, the absence of UNDOCUMENTED will be used instead. Since requiring a documentation category for undocumented options doesn't make sense, list that as one of the OptionDocumentationCategories, but list HIDDEN and INTERNAL as part of OptionMetadata. These options should list UNDOCUMENTED as their category. PiperOrigin-RevId: 161515674
* Add the default category and tag to all options.Gravatar ccalvarin2017-06-28
| | | | | | | | Move the default from the annotation to every mention. This makes the incompleteness explicit. Will add the defaults to test targets in a separate change. Once all dependencies are cleaned up, the Option annotation will no longer allow options without the documentationCategory or effectTag, to prevent new options being added without categories while we migrate to the new option categorization. PiperOrigin-RevId: 160281252
* Use getopt to parse process-wrapper's command-line.Gravatar philwo2017-06-22
| | | | | | | | This will allow us to add new and optional flags like selecting a strategy used to spawn / wait for the child process. No one except Bazel should be calling "process-wrapper" and I couldn't find any references, so this breaking change should be fine. PiperOrigin-RevId: 159685867
* Rewrite StandaloneSpawnStrategy to use LocalSpawnRunnerGravatar ulfjack2017-06-19
| | | | PiperOrigin-RevId: 159221067
* Fix local execution to not throw an exceptionGravatar ulfjack2017-06-14
| | | | | | | | The SpawnInputExpander returns null for empty files, but the ActionInputPrefetcher does not expect null values, and implementations may throw NPE. PiperOrigin-RevId: 158960425
* Extract the MacOS/XCode env rewrite logic into lib.exec.appleGravatar ulfjack2017-06-13
| | | | | | | | | | | | | | | | | | | Also add an interface to allow injecting that logic into LocalSpawnRunner; this is in preparation for rewriting StandaloneSpawnStrategy to use LocalSpawnRunner. At the same time, this reduces the dependencies from exec / standalone to rules.apple, which is a prerequisite for micro-Bazel. There's a small semantic change hidden here - we now only set the new XCodeLocalEnvProvider if we're actually running on Darwin, so we no longer fail execution on non-Darwin platforms if XCODE_VERSION_OVERRIDE or APPLE_SDK_VERSION_OVERRIDE is set. As a result, I moved the corresponding test from StandaloneSpawnStrategyTest to the new XCodeLocalEnvProviderTest. While I'm at it, also open source DottedVersionTest and CacheManagerTest. PiperOrigin-RevId: 158829077
* Add some tests to the LocalSpawnRunner, and make some small tweaksGravatar ulfjack2017-06-08
| | | | | | | | | - Only manually check the timeout if the process wrapper is not used - Set the timeout correctly; the process API uses milliseconds - flush the error output stream after writing - return SIGALRM as exit code for timeout cases PiperOrigin-RevId: 158374246
* Add a tag to ExecutionRequirements that allows disabling local prefetchGravatar ulfjack2017-06-06
| | | | | | | Remove SpawnExecutionPolicy.shouldPrefetchInputsForLocalExecution in favor of a static helper method in Spawns. PiperOrigin-RevId: 158119993
* Implement logging in LocalSpawnRunnerGravatar ulfjack2017-04-18
| | | | | | | | This is a prerequisite for Google to use it; there's no consistent logging strategy yet, but Google internally requires logging, and it needs to be somewhat backwards compatible. PiperOrigin-RevId: 153454160
* Simplify LocalSpawnRunnerGravatar ulfjack2017-04-18
| | | | PiperOrigin-RevId: 153444516
* Split out option usage restriction from option category.Gravatar ccalvarin2017-04-18
| | | | | | | | | These are two different concepts. Do not remove category overload compatibility in this CL, to keep this change limited to converting the current uses of category. With some flyby formatting fixes on affected OptionsBases. RELNOTES: None. PiperOrigin-RevId: 153390002
* LocalSpawnRunner: on Windows, don't use the process wrapper for nowGravatar ulfjack2017-04-07
| | | | | This is ported from StandaloneSpawnStrategy. PiperOrigin-RevId: 152493898
* Open source LocalSpawnRunnerGravatar ulfjack2017-04-07
The LocalSpawnRunner is a non-sandboxed local execution implementation, which will replace the current StandaloneSpawnStrategy. The code has been around for a long time and has seen a lot of bugfixes. It also supports local prefetching, which is required for Google. I have a follow-up change to make it support Windows, so it's not a drop-in replacement for StandaloneSpawnStrategy yet. PiperOrigin-RevId: 152486973
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-07 07:41:12 +0000