aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/runtime/BlazeRuntime.java
Commit message (Collapse)AuthorAge
* Add experimental option to do add local cpu usage values to the JSON profile.Gravatar twerth2018-08-14
| | | | | RELNOTES: None PiperOrigin-RevId: 208646319
* Remove unused workspace variable.Gravatar twerth2018-08-09
| | | | | RELNOTES: None PiperOrigin-RevId: 208010973
* Add a option to disable idle gc.Gravatar Benjamin Peterson2018-08-08
| | | | | | | | | | | | | | | If a Bazel server is idle for 10 seconds, it unconditionally triggers a full-scale Java GC via System.gc(). This behavior doesn't have clear benefits and causes Bazel to steal resources from whatever the user does after invoking Bazel. This CL adds a startup option, --idle_server_tasks, to toggle the idle GC behavior. Also, add some logging for when idle GC is enabled, so it's easier to evaluate its effects. Example of logging: ``` 180718 17:43:04.609:I 247 [com.google.devtools.build.lib.server.IdleServerTasks.lambda$idle$0] [Idle GC] used: 157MB -> 15MB, committed: 421MB -> 422MB ``` Fixes https://github.com/bazelbuild/bazel/issues/5589. Closes #5628. PiperOrigin-RevId: 207869996
* Add option to post ProfileStartedEvent containing the profile's path.Gravatar twerth2018-07-31
| | | | | RELNOTES: None PiperOrigin-RevId: 206741115
* Add an option to compress the JSON trace profile.Gravatar twerth2018-07-27
| | | | | | | chrome://tracing is able to load gzipped profiles out of the box. RELNOTES: None PiperOrigin-RevId: 206308018
* bep: introduce BuildEventArtifactUploaderFactoryGravatar buchgr2018-07-09
| | | | | | | | | | | | | | | | There can be multiple BuildEventTransports active at the same time and we need to ensure that each transport gets its own BuildEventArtifactUploader as these transports might have different lifecycles. We do that by introducing another level of indirection via the BuildEventArtifactUploaderFactory. BlazeModules now register a factory object instead of an uploader. In addition, the BuildEventArtifactUploader gets a shutdown() method that allows to free any resources associated with it. PiperOrigin-RevId: 203752092
* Replace instances of Blaze with Bazel.Gravatar Dan Fabulich2018-07-05
| | | | | | | PiperOrigin-RevId: 203300374 Change-Id: Iaa47f870ab2e0cad40a202aad2c7f9430f73c856 PiperOrigin-RevId: 203407392
* Add a bit more profiler coverageGravatar ulfjack2018-06-15
| | | | | | | | | | | | | | | | | | The intent is that the main thread has ~zero gaps in the profile (though there may still be small gaps due to the time between one try block and an immediately subsequent try block). We need to be careful not to wrap markPhase calls (or methods that call markPhase) in try blocks for the profiler - the Profiler requires that all markPhase calls happen at top level, and throws an exception if not. This should not have any performance impact - all of these are once per build, or at most once per module per build, and we don't expect a very large number of modules (and if we see an increasing number, we need to change the module API to not have to call every single module, but only those that are actually interested in certain events, maybe with an EventBus-based setup). PiperOrigin-RevId: 200712677
* Move the BlazeRuntime.initProfiler call to BlazeCommandDispatcherGravatar ulfjack2018-06-15
| | | | | | | | | | The intend is to move the profiler setup earlier in the startup sequence, so we can get more profiler coverage. Ideally, we'd setup the profiler immediately after getting the client call, but this is not currently possible due to the requirement that initialization and shutdown happen in pairs (otherwise the next command invocation crashes with an exception from the Profiler). PiperOrigin-RevId: 200707029
* Refactor BlazeRuntime.initProfilerGravatar ulfjack2018-06-15
| | | | | | | - reduce the parameters to the minimum required - change to void, move additional setup into the method PiperOrigin-RevId: 200692559
* Add a mechanism for build event protocol events to upload filesGravatar ulfjack2018-06-15
| | | | | | | | This should be a no-op, mostly replacing PathConverter with BuildEventArtifactUploader, since none of the implementations perform any upload yet. PiperOrigin-RevId: 200685325
* Fix Profiler setup to actually enable the Json formatGravatar ulfjack2018-06-14
| | | | PiperOrigin-RevId: 200518973
* Add support for the Json trace file formatGravatar ulfjack2018-06-12
| | | | | | | | | | | | | | | | | Add a --experimental_generate_json_trace_profile option that puts a file into the output base (or uses --profile if set). There are still a lot of problems with this. - unexplained holes - way too many threads - nonsensical event titles - too many detail events, too little overview - it may also cause unnecessary load - it silently overwrites the existing file on subsequent invocations The format is documented here: goo.gl/oMZPLh PiperOrigin-RevId: 200259431
* Refactor profilerGravatar ulfjack2018-06-12
| | | | | | | | | | | | | | | | | | | | | | | | | | - move the save method to an inner class - don't use a timer, use a blocking queue instead - add a format enum (in anticipation of adding a json output format) - update the test to use an in memory buffer, and avoid FoundationTestCase Compared to the original https://github.com/bazelbuild/bazel/commit/15b8c259db111012b4642287172cb4d1d82151f3, it contains these changes: - Make it so we don't create a queue if we are not going to write any data! The queue is now owned by the writer, and if there is no writer, there is no queue. This was causing a memory regression because slowest task profiling is enabled by default, in which case the profiler is started with no output file. In that case, there's no thread that is emptying the queue, but the queue was still created by default. - add additional tests for slowest task and histogram handling; these also provide coverage for the case where the profiler is started without an output stream - move all the writer thread handling into the inner class - make writer access thread-safe - add a bunch of documentation PiperOrigin-RevId: 200212978
* Automated rollback of commit 15b8c259db111012b4642287172cb4d1d82151f3.Gravatar jmmv2018-06-11
| | | | | | | | *** Reason for rollback *** Breaks internal performance tests. PiperOrigin-RevId: 200103033
* Refactor profilerGravatar ulfjack2018-06-11
| | | | | | | | | - move the save method to an inner class - don't use a timer, use a blocking queue instead - add a format enum (in anticipation of adding a json output format) - update the test to use an in memory buffer, and avoid FoundationTestCase PiperOrigin-RevId: 200065404
* Clean up Profiler a bitGravatar ulfjack2018-06-08
| | | | PiperOrigin-RevId: 199849102
* Port bazelrc flags into their own module.Gravatar ccalvarin2018-05-16
| | | | | | | | | This way the documentation logs them correctly as named "bazel," and describes the correct behavior. Needed for #4502. RELNOTES: None. PiperOrigin-RevId: 196837021
* Move --track_incremental_state and --keep_state_after_build to ↵Gravatar felly2018-05-15
| | | | | | CommonCommandOptions to better facilitate certain kinds of benchmarking. PiperOrigin-RevId: 196695342
* Use the local JDK as the default target javabaseGravatar cushon2018-05-14
| | | | | | and continue to use the embedded JDK as the default host_javabase. PiperOrigin-RevId: 196471714
* Add hook to inform SkyframeExecutor that build is finished.Gravatar janakr2018-04-24
| | | | PiperOrigin-RevId: 194099006
* Add BazelCrashUtils to make it easier to crash the JVM when uncaught ↵Gravatar janakr2018-04-20
| | | | | | exceptions are detected in async worker threads. PiperOrigin-RevId: 193725871
* A slight refactoring of RunCommand.Gravatar lberki2018-04-17
| | | | | RELNOTES: None. PiperOrigin-RevId: 193203414
* Remove some unnecessary params.Gravatar nharmata2018-04-02
| | | | | RELNOTES: None PiperOrigin-RevId: 191354727
* ArtifactRoot correctly switches output base when serializing.Gravatar shahan2018-03-28
| | | | PiperOrigin-RevId: 190779535
* Victory lap: Remove all code that used to support the three-argument form of ↵Gravatar lberki2018-03-23
| | | | | | | vardef(). RELNOTES: None. PiperOrigin-RevId: 190196933
* Add a DefaultBuildOptions specifying default build flags that may differ ↵Gravatar mjhalupka2018-03-22
| | | | | | from the provided defaults in Options classes. These are used to create BuildOptionsDiffForReconstruction, which lets us store only the diffs in our BuildConfigurationValue.Keys. PiperOrigin-RevId: 190117455
* Convert SkylarkActionFactory methods to use @SkylarkCallable instead of ↵Gravatar cparsons2018-03-22
| | | | | | | @SkylarkSignature RELNOTES: None. PiperOrigin-RevId: 190073508
* Make the builtins registry thread-safeGravatar brandjon2018-03-06
| | | | | | | | | | | | | | It was previously assumed that safety wasn't needed because 1) all builtins should be registered in static initializer blocks, and 2) all retrievals should occur during Skylark evaluation, after static initialization completes. It turns out these assumptions aren't actually true (Who would've thunk it!). SkylarkActionFactory has been observed to be initialized as late as analysis time, and retrievals occur as early as constructing a PackageFactory (when scanning the native module). The failure mode is particularly ugly: Random Skylark method lookups will fail non-deterministically. This change guards against this by making the builtins registry implement a form of freezing. Before freezing, reads and writes are allowed and are synchronized. After freezing, only reads are allowed and they are unsynchronized for performance. BlazeRuntime is responsible for flipping the bit, and for ensuring classes like SkylarkActionFactory run their initialization by that point. Unit tests don't need to worry, since they just stay unfrozen and synchronized throughout. RELNOTES: None PiperOrigin-RevId: 188080136
* Add functionality to MemoryProfiler to do multiple garbage collections at ↵Gravatar janakr2018-02-28
| | | | | | the end of the build in an effort to get an accurate measurement of used memory. PiperOrigin-RevId: 187337487
* Expose --output_user_base to the Bazel server processGravatar Klaus Aehlig2018-02-27
| | | | | | | | | | | ...so that it can use that path to compute other directories in the output user base, in particular the default location for caches. The first cache we will add is the hash-index cache for downloads of external archives, but a spawn cache might be added later in the output user base as well. Change-Id: I24b1c33235c8f76ec008ecb1789163de2b2a45be PiperOrigin-RevId: 187164275
* Accept Durations in the CriticalPathComponent and Profiler.Gravatar ccalvarin2018-02-22
| | | | | RELNOTES: None. PiperOrigin-RevId: 186658512
* Move BinTools to lib.execGravatar ulfjack2018-02-15
| | | | | | | It's never used during analysis, it shouldn't be used during analysis, so let's not make it available during analysis. PiperOrigin-RevId: 185808384
* Delete the now-useless interfaces ServerCommand and CommandExecutor.Gravatar lberki2018-02-06
| | | | | | | | | We apparently don't have any other implementations of these interfaces than BlazeCommandDispatcher, so let's not have them at all; we can always put back an interface with the exec() method if need be. RELNOTES: None. PiperOrigin-RevId: 184698573
* Move BlazeRuntime/BlazeCommandDispatcher shutdown to where they are created.Gravatar lberki2018-02-06
| | | | | RELNOTES: None. PiperOrigin-RevId: 184692000
* Remove ShutdownBlazeServerException in favor of indicating that the server ↵Gravatar lberki2018-02-06
| | | | | | | should be shut down in BlazeCommandResult. RELNOTES: None. PiperOrigin-RevId: 184678994
* Add a "direct" mode to "blaze run" that makes the process being run a directGravatar lberki2018-02-05
| | | | | | | | | | | | | | child of the process where the Blaze client itself was run. Limitations: - Untested on Windows; it should work because ExecuteProgram() is implemented there, too, but since Windows doesn't support exec(), there is at least one process in between Progress towards #2815. RELNOTES[NEW]: The new "--direct_run" flag on "blaze run" lets one run interactive binaries. PiperOrigin-RevId: 184528845
* Add option to optionally wipe state at the end of a build.Gravatar ccalvarin2018-01-22
| | | | | | | This will serve as an alternative to --batch, leaving behind a server without state from the previous build. RELNOTES: Introduces --[no]keep_state_after_build PiperOrigin-RevId: 182778500
* Remove redundant words in logging message.Gravatar nharmata2017-12-08
| | | | | RELNOTES: None PiperOrigin-RevId: 178407067
* Add ActionKeyContext to Action#getKey.Gravatar tomlu2017-11-29
| | | | | | | This key context can be used by actions to share partial key computations, for instance when computing MD5s for nested sets. RELNOTES: None PiperOrigin-RevId: 177359607
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Thread FileSystem through to a single Path#delete call site.Gravatar tomlu2017-10-30
| | | | | | This requires a fairly large amount of changes to fundamental objects like BlazeRuntime, Executor, and so on, as well as changing a lot of test code to thread the file system through. I expect future CLs to be much smaller. PiperOrigin-RevId: 173678144
* Track Option placement within a priority category.Gravatar ccalvarin2017-10-18
| | | | | | | | | An option has precedence over previous options at the same enum-valued priority. Track its placement in this ordering explicitly. This will allow after-the-fact expansion of expansion options such that they correctly take precedence or not compared to other mentions of the same flag. This is needed to fix --config's expansion. RELNOTES: None. PiperOrigin-RevId: 172367996
* Make UnionFileSystem accept all paths Bazel can throw at it.Gravatar ccalvarin2017-09-27
| | | | | | | | | Instead of relying on a character-by-character StringTrie, segment paths based on PathFragments. This means UnionFS can accept any path that Bazel stores internally, removing the ASCII limitations. This also means removing the ability to have a filesystem bound for sub-PathFragments, /foo/barbar, /foo/barqux could have the same filesystem bound at /foo/bar. This feature was tested for when a use case was envisioned, but it was never used, so removing it is safe. RELNOTES: None. PiperOrigin-RevId: 170054656
* Remove product name precomputed value.Gravatar janakr2017-09-25
| | | | PiperOrigin-RevId: 169723724
* Remove deepExecRoot from BlazeDirectories. It is now the default. Also ↵Gravatar janakr2017-09-14
| | | | | | inline constructor that did ServerDirectories creation inline: don't pollute production code for tests' convenience. PiperOrigin-RevId: 168652349
* Replace referrals to options by their name to option definitions.Gravatar ccalvarin2017-09-12
| | | | | | | Now that we have a standard way of referring to an option, remove all of the places that we were referring to them by their name. Since options can have multiple names, this is more clear and provides the additional information needed to understand the option. It also stops the habit of requesting unqualified strings, which was hard to read. RELNOTES: None. PiperOrigin-RevId: 168254584
* Move Subprocess.Factory to a top-level classGravatar ulfjack2017-09-08
| | | | | | | | | | | Also move the implementation of FutureCommandResult to a top-level class. This is in preparation for significantly simplifying the shell library. The plan is to remove the Subprocess abstraction, and have lower-level implementations implement the much simpler FutureCommandResult interface instead. PiperOrigin-RevId: 167844736
* More BUILD file refactorings.Gravatar philwo2017-09-06
| | | | | | | | | Split collect, concurrent, vfs, windows into package-level BUILD files. Move clock classes out of "util", into their own Java package. Move CompactHashSet into its own Java package to break a dependency cycle. Give nestedset and inmemoryfs their own package-level BUILD files. PiperOrigin-RevId: 167702127
* Fix assorted ErrorProne warnings.Gravatar lberki2017-09-06
| | | | | RELNOTES: None. PiperOrigin-RevId: 167574104
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-28 19:30:05 +0000