aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/worker/WorkerModule.java
Commit message (Collapse)AuthorAge
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Add the workspace name to the sandbox path for sandboxed workers.Gravatar philwo2017-08-18
| | | | PiperOrigin-RevId: 165571541
* Refactor persistent workers to use SpawnRunner.Gravatar Benjamin Peterson2017-08-11
| | | | | | | | | | | | | | | | | | | | | | | | | | Change the persistent worker spawn strategy to extend AbstractSpawnStrategy and put the actual logic into WorkerSpawnRunner. WorkerTestStrategy is unaffected. I had to extend SpawnPolicy with a speculating() method. Persistent workers need to know if speculation is happening in order to require sandboxing. Additionally, I added java_test rules for the local runner tests and worker tests. See https://github.com/bazelbuild/bazel/issues/3481. NOTE: ulfjack@ made some changes to this change before merging: - changed Reporter to EventHandler; added TODO about its usage - reverted non-semantic indentation change in AbstractSpawnStrategy - reverted a non-semantic indentation change in WorkerSpawnRunner - updated some internal classes to match - removed catch IOException in WorkerSpawnRunner in some cases, removed verboseFailures flag from WorkerSpawnRunner, updated callers - disable some tests on Windows; we were previously not running them, now that we do, they fail :-( Change-Id: I207b3938f0dc84d374ab052d5030020886451d47 PiperOrigin-RevId: 164965398
* Simplify BlazeModule.beforeCommandGravatar ulfjack2017-06-14
| | | | | | | | | | | Don't pass the Command annotation explicitly, but add it to CommandEnvironment instead; most modules don't need it in the first place, so it was a lot of boilerplate for not much. Also change it so that the command is passed to the constructor. Add some documentation to the beforeCommand method. PiperOrigin-RevId: 158847128
* Fix #2769: Worker processes should be killed on 'bazel clean'.Gravatar philwo2017-04-12
| | | | PiperOrigin-RevId: 152799488
* Move some classes to lib.exec in preparation for layering enforcementGravatar Ulf Adams2016-12-21
| | | | | | | | | | | | This change is part of the mu-bazel effort, which aims to build a minimally useful Bazel binary with most extraneous functionality removed. As part of that, we want to enforce layering of packages. In particular, lib.actions must not depend on lib.rules or lib.exec. lib.rules must not depend on lib.exec. Moving these classes is a necessary step to enforce that layering. -- PiperOrigin-RevId: 142668172 MOS_MIGRATED_REVID=142668172
* Refactor the module API to use the builder pattern for executor creation.Gravatar Ulf Adams2016-11-02
| | | | | | | This significantly simplifies several of our modules. -- MOS_MIGRATED_REVID=137713119
* Add support for sandboxing persistent workers.Gravatar Philipp Wollermann2016-09-28
| | | | | -- MOS_MIGRATED_REVID=134523222
* Split BlazeModule.getCommandOptions into two; unify all implementations.Gravatar Ulf Adams2016-06-23
| | | | | | | | | | | Several modules now explicitly add common command options. Of the remaining ones, most add options to the build command, except one, which adds options to query. They now all use the canonical implementation. Also updated the documentation to clarify what this method actually does. -- MOS_MIGRATED_REVID=125560058
* workers: Improve verbose messages when workers are restarted. Prevent ↵Gravatar Philipp Wollermann2016-05-24
| | | | | | | | | | | | | | | | | killing of broken workers in the background, when no build is currently running, because we can only alert the user about what's going on while a build is currently running. If you're a developer, this will probably help you when debugging a new worker. Some messages you might see with --worker_verbose specified: INFO: Worker configuration has changed, restarting worker pool... WARNING: Javac worker (id 6) can no longer be used, because its files have changed on disk [13ce6c10546243b6b4ea2334dd3f8a55705e4c078cedd0e746d7ad7cee082e9a -> 9731223c50bab5c1f20d60d1a4967cf2726840727b6a639b6f6425547a9b4a10]. WARNING: Javac worker (id 4) can no longer be used, because its process terminated itself or got killed. -- MOS_MIGRATED_REVID=123093071
* workers: When a WorkerFactory is instantiated, first remove old worker logs ↵Gravatar Philipp Wollermann2016-05-24
| | | | | | | that belong to a prior running Bazel server. -- MOS_MIGRATED_REVID=123092141
* Fix #837: worker_verbose flag has no effect.Gravatar Philipp Wollermann2016-05-19
| | | | | | | Due to reusing an old Reporter instead of grabbing the current one for each build, verbose messages were lost and not printed to the console. Also adds a test for this feature. -- MOS_MIGRATED_REVID=122639383
* Make worker more defensive about NPEsGravatar Kristina Chodorow2016-04-01
| | | | | | | | | I was messing with filesystem dirtiness checkers and got an NPE, and I guess it was so early in the build that the buildRequest didn't have WorkerOptions yet, so I got a NPE from WorkerModule.buildComplete masking the actual NPE. -- MOS_MIGRATED_REVID=118780059
* Move most BlazeRuntime methods related to workspace directories to CommandEnv.Gravatar Ulf Adams2016-03-30
| | | | | -- MOS_MIGRATED_REVID=118561661
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* workers: Some tweaks to the WorkerPool config.Gravatar Philipp Wollermann2015-09-21
| | | | | -- MOS_MIGRATED_REVID=103541116
* Introduce CommandEnvironment and pass it instead of BlazeRuntime.Gravatar Ulf Adams2015-09-15
| | | | | | | | The very long term plan is to make BlazeRuntime immutable, so that we can run multiple commands in the same server. -- MOS_MIGRATED_REVID=103080946
* workers: A multitude of bug fixes and improved logging.Gravatar Philipp Wollermann2015-09-14
| | | | | | | | | | | | | | | | | | | | | | | I know this should have been split up, but I was a bit on fire today and did it all in one go ^^; Fixes spurious "Stream closed: Stream closed" errors, by noticing dead workers and retrying with a fresh one. (Configurable with the --worker_max_retries flag.) Fixes an "IllegalArgumentException" when a non-worker compatible Spawn is given to the strategy. We fall back to StandaloneSpawnStrategy now. Redirect the stderr of worker processes to separate log files in a common sub-directory and print a message that tells you about the location on worker start-up for easier debugging. The log can be found in <output_base>/worker-logs/*.log. Adds the mnemonic of the Spawn to log messages and the log filename. Adds verbose messages on worker start-up and shutdown. (Enable it with --worker_verbose!) Shuts down the worker pool after a build finished by default, until we sort out one last remaining correctness issue. This also conserves resources, though makes incremental builds a bit slower. Want the maximum performance anyway? Try --experimental_workers_keep_running. Adds stack traces to errors that are caused by buggy workers to aid development. Fixes weird dupli..tripli..quadruple error messages ("Compiling failed: Stream closed: Stream closed: Stream closed: Stream closed."). -- MOS_MIGRATED_REVID=102983853
* Only run the worker if the build is incremental. Gravatar Han-Wen Nienhuys2015-06-26
| | | | | | | This feature is tunable through --worker_max_changed_files flag. If unspecified, all builds are considered incremental. -- MOS_MIGRATED_REVID=96968367
* Implement persistent worker processes and a spawn strategy that uses them.Gravatar Philipp Wollermann2015-06-10
This will be used by the persistent JavaBuilder, which improves performance of Java compilation by 4x due to profiting from JVM JIT optimizations and not having to relaunch the JVM for every spawn. It is completely generic though, so as long as a tool (ProGuard? Dexer? Whatever.) conforms to the Worker process protocol, it can use the new spawn strategy. -- MOS_MIGRATED_REVID=95527132