aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/cpp/blaze.cc
Commit message (Collapse)AuthorAge
* Add a --server_javabase startup option as an alias for the LHS --host_javabaseGravatar cushon2018-07-25
| | | | | | | | | | to avoid confusion between the LHS and RHS host_javabases. The LHS --host_javabase option should be considered deprecated and will eventually be removed. RELNOTES: Rename the startup flag --host_javabase to --server_javabase to avoid confusion with the build flag --host_javabase PiperOrigin-RevId: 206015757
* Fix block_for_lock.Gravatar ccalvarin2018-07-23
| | | | | | | | | | | | | | | | | | | | | | | | | | On two fronts: First, it should follow standard command line semantics. Second, it should work as intended: --noblock_for_lock means the client will not wait for another command to finish, but will exit eagerly. It can be useful for preventing hanging in applications that are non-interactively calling bazel. It should have standard startup-option semantics: the default value is accepted as a no-op or can be provided to override a previous value. The next issue involves 2 different locks - the client lock, and the server-side command lock. This duality exists because we would like, one day, to be able to run certain commands, like info or help, at the same time, so multiple commands would need specialized locks that allow some duality but blocks others. This can only be done at the server level, so as soon as the client gets the "we're connected" grpc message from the server, it releases the client lock and lets the server manage multiple requests. There are basically 3 possible states that are relevant to this option: 1) no other client is active, so no one holds the client lock or the command lock - the server can be used, shutdown or started as needed. - no blocking, but no need to block, either, so we're safe 2) another client (client1) holds the lock, but it is currently using a server that we want to reuse. If client1 still holds the client lock, we fail fast. Same thing if client1 is holding the server-side lock: we will exit gracefully when the BlazeCommandDispatcher responds with a failure. 3) client1 holds the lock but its server cannot be reused. (batch clients also fall into this category, as there is no server to reuse - but in this case, the client lock is still in play). However, for server mode, this is broken - the following happens: - Server is occupied with client1's request, holds the command lock - client2 wants to restart the server, so sends the old server a "shutdown" command - the BlazeCommandDispatcher says - nuh-uh, this is busy, and you said you didn't want to wait for the lock - client2 absorbs this response - waits (blocks...) - for a minute - then force shuts-down the old server. So we had 2 problems - we block, and we shutdown a server that we truly intended to keep going. Now, if the server responds saying another action is using it, the client will exit correctly, and leave the old server to do its thing. RELNOTES: None. PiperOrigin-RevId: 205671817
* blaze.cc: add missing fflush(3) before execve(2)Gravatar Klaus Aehlig2018-07-23
| | | | | | | | | | When calling `bazel run` the command itself is executed by the client. As an execve(2) replaces the program image, including all buffered IO, flush all streams first. This will ensure that the "Running command line" message is actually printed. Change-Id: Ie18185bac4ed82a2725c75f97d3c64bd3003690b PiperOrigin-RevId: 205652760
* jdk: use parallel old gc and disable compact stringsGravatar buchgr2018-07-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When switching to JDK9 we regressed on java build performance by about ~30%. We found that when using parallel gc instead of G1 and disabling compact strings for JavaBuilder, the java build performance is "only" 10% slower. We additionally found JDK9 to have a significantly higher startup time. This impacts the performance of tools like javac and tubine and we believe that this accounts for most of the remaining overhead that can't be explained by disabling G1 and compact strings. java8 -version: 80ms java9 -version: 140ms java10 -version: 110ms Additionally, we found that the number of modules shipped with the JDK have a direct effect on the startup time. When building Java 10 with only the 9 modules required by Bazel we find that the startup time reduces to 80ms (from 110ms) which is on par with Java 8. We thus expect the regression to be fixed by a future migration to Java 10, which should be done in one of the next Bazel releases. == Some benchmark results == https://github.com/google/protobuf $ bazel build :protobuf_java Bazel 0.15.2: 4.2s Bazel 0.16.0-rc2: 5.2s This Change: 4.2s https://github.com/jin/android-projects/tree/master/java_only $ bazel build :module0 Bazel 0.15.2: 8.2s Bazel 0.16.0-rc2: 11.5s This Change: 9.1s RELNOTES: None PiperOrigin-RevId: 205647957
* Fix race condition between server startup and command port reading.Gravatar twerth2018-07-18
| | | | | RELNOTES: None PiperOrigin-RevId: 205079775
* Windows,Bazel client: multithreaded file writingGravatar Laszlo Csomor2018-07-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Bazel client on Windows now writes extracted binaries to disk in parallel. On all other systems it writes them serially (as before). This change makes blaze.cc:ActuallyExtractData() about 3x faster when using a HDD. (In previous experiments I saw no speedup with multi-threaded writing on machines with an SSD.) The Windows-specific code uses the native Threadpool API of Windows, creating a pool of at least 8 and at most 16 threads. (This seems to be a good balance between speed and thread count.) The OS manages everything about the pool; Bazel submits callbacks and the pool executes them asynchronously. blaze.cc:ActuallyExtractData() speed, before: - Windows: 6.48s (avg) / 6.38s (median) - Linux (Debian): 4.78s (avg) / 4.79s (median) blaze.cc:ActuallyExtractData() speed, after: - Windows (8-16 threads): 2.05s (avg) / 2.01s (md) - Windows (1 thread): 5.77s (avg) / 5.74s (median) See https://github.com/bazelbuild/bazel/issues/5444 Change-Id: I7211f3d28eb8b9837352c16ff8df0411d5a9ebe1 Closes #5600. Change-Id: I7a74d62a563c92948a4dfa8ad5ac83eae018db10 PiperOrigin-RevId: 204891217
* Automatic ClangTidyReadability code cleanup.Gravatar Googler2018-06-29
| | | | PiperOrigin-RevId: 202644968
* Bazel,client: actually cache directory creationsGravatar Laszlo Csomor2018-06-27
| | | | | | | | | | | | | | | | | | | | Commit https://github.com/bazelbuild/bazel/commit/f5043d6831ea1c266104363b4e8911eb97f96fbc was incorrect in that it cached the file names, not the directory names. This commit fixes that. I verified that the number of calls to ExtractBlazeZipProcessor::Process is greater than the calls to MakeDirectories within (1038 vs. 172 on Linux). See https://github.com/bazelbuild/bazel/issues/5444 Change-Id: I314bdc9337c9782a5ceaed7aac785a552b222b1f Closes #5484. Change-Id: I314bdc9337c9782a5ceaed7aac785a552b222b1f PiperOrigin-RevId: 202314400
* Windows,client: extract embedded binaries fasterGravatar Laszlo Csomor2018-06-25
| | | | | | | | | | | | | | | | | | | | When extracting embedded binaries, the client now caches which directories it has already created and won't attempt creating them again. This saves some time on Windows: from 16.3 sec on average down to 13.2 sec. (n=10 runs, always starting Bazel with a new --output_user_root and shutting down afterwards.) On Linux I see only a marginal speedup, not significant enough to claim credit for it. :) See https://github.com/bazelbuild/bazel/issues/5444 Closes #5448. PiperOrigin-RevId: 201933181
* Windows,Bazel client: check embedded tools fasterGravatar Laszlo Csomor2018-06-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | The Bazel client on Windows is now 50% faster to check the embedded tools than it was before. Results: - Linux: 20 ms -> 6 ms - Windows: 294 ms -> 133 ms Measurements were done with n=10 runs and a hot server, using blaze::GetMillisecondsMonotonic(). Previously the client performed the same tasks multiple times while trying to determine if a path was a good extracted binary. (E.g. converted the path to Windows format multiple times, checked if it was a directory twice, opened the path twice.) Now the client performes these tasks only once, e.g. it converts path once and stats only once. See https://github.com/bazelbuild/bazel/issues/5444 Closes #5445. PiperOrigin-RevId: 201913758
* PiperOrigin-RevId: 201390253Gravatar Googler2018-06-20
|
* do not start server if shutdown requested (+ not in batch mode)Gravatar Googler2018-06-19
| | | | PiperOrigin-RevId: 201144030
* Remove the connect-to-server dots at server startup.Gravatar ccalvarin2018-06-15
| | | | | | | | Replace with an update at most every 10 seconds if we are still trying to connect. TESTED: Tested manually that this does print every 10 minutes if the server is prevented from connecting. RELNOTES: None. PiperOrigin-RevId: 200764279
* update bazel's embedded jdk to jdk9Gravatar cushon2018-06-11
| | | | | | | | | | | - Updates the embedded JDK to Azul Zulu 9.0.7 - All integration tests use Bazel with the embedded JDK Also updated: http://storage.googleapis.com/bazel-mirror/openjdk/index.html Closes #5312, #5314, #5315 PiperOrigin-RevId: 200055008
* Move path-manipulation functions to own library file.Gravatar ccalvarin2018-06-05
| | | | | | | | | | | Leave functions that make file accesses in the file library, and general blaze utilities in the blaze_util file, but move the functions that boil down to string manipulation and path formatting to their own file. (With the exception of getCWD, since absolute path syntax is relevant here.) Doing this largely to consolidate all Windows path control into a single place, so that it's easier to notice inconsistencies. For instance, ConvertPath currently makes Windows paths absolute, but not Posix paths, and MakeAbsolute relies on this behavior. In addition, JoinPath assumes Posix path syntax, which leads to some odd looking paths. These will be fixed in a followup change. (Found these issues while working on #4502, trying to fix the windows-specific system bazelrc.) RELNOTES: None. PiperOrigin-RevId: 199368226
* Set no_proxy=localhost when an http_proxy is detected, instead of deleting ↵Gravatar ruperts2018-06-04
| | | | | | | | | the http_proxy value. Also change the environment for the client and the server in this case, instead of only changing the server's environment. RELNOTES: None. PiperOrigin-RevId: 199152406
* Add --ignore_all_rc_files startup options.Gravatar ccalvarin2018-05-15
| | | | | | | | | | | | | This overrides --bazelrc and --[no]master_bazelrc regardless of order. Like --bazelrc and --[no]master_bazelrc, it cannot be mentioned in an rc file, this would be a contradiction. This flag is useful for testing, and for having a version-agnostic way to turn off all rc files, such as in the canonical command line reporting. Now that blazerc and bazelrc are separate, this is necessary. If explicit values for --bazelrc or --master_bazelrc are provided which are now ignored, Bazel will warn the user. #4502 Alternatives considered - We could avoid this flag but would need to have some well-documented, reusable list of the startup flags that effectively come to the same effect. This would be necessary in our integration tests and in the CommandLineEvent and other places where rc files need to be completely disabled for correctness. We decided that this startup option was more straightforward and usable for both users and Bazel devs: it shouldn't be used when more fine-grained control is needed, but provides warnings if users are likely to be confused by the outcome. RELNOTES: None. PiperOrigin-RevId: 196750704
* Use the local JDK as the default target javabaseGravatar cushon2018-05-14
| | | | | | and continue to use the embedded JDK as the default host_javabase. PiperOrigin-RevId: 196471714
* Automated rollback of commit 52356e76a02067e9dc79b469e9b97bd017d077ab.Gravatar jingwen2018-04-27
| | | | | | | | | | | *** Reason for rollback *** Breaks Bazel CI pipeline (pre and postsubmits) on all platforms: https://buildkite.com/bazel/bazel-bazel/builds/1785 Fixes https://github.com/bazelbuild/bazel/issues/5113 RELNOTES: None. PiperOrigin-RevId: 194620643
* Automatic code cleanup.Gravatar Googler2018-04-27
| | | | PiperOrigin-RevId: 194602500
* Keep LANG/LANGUAGE and related environment variables for the process spawned ↵Gravatar lberki2018-04-18
| | | | | | | by "blaze run --direct_run". RELNOTES: None. PiperOrigin-RevId: 193391379
* Remove die() and replace it with BAZEL_DIE, part of the logging framework.Gravatar ccalvarin2018-04-06
| | | | | | | This will mean the messages will make it to the right output stream. RELNOTES: PiperOrigin-RevId: 191925662
* Make the client's java startup command line list all startup options.Gravatar ccalvarin2018-03-30
| | | | | | | | | | | We expect that the client passes all startup options to the server, default or explicit. The server's listing of default values should not matter. Yet for a number of these options, the default value in the server was relied upon, because the server command line was not constructed with the client's default value included. Fix visible cases of this, long term this should be tested for, so the invariant is not broken again. This has been the documented expectation for a long time, but a number of violations have crept up over time. Update the comments that lead to this expectation to be more realistic. Add debug statement that shows which options are changed when startup options cause the server to be restarted. The detailed logs will only be seen if --client_debug is set to TRUE. RELNOTES: None. PiperOrigin-RevId: 191066983
* Remove pdie.Gravatar ccalvarin2018-03-29
| | | | | | | pdie and die are pretty similar, pdie just adds the errno string or equivalent from GetLastErrorString(). Make this explicit. This makes message formatting more clear in preparation for moving these all to BAZEL_LOG. RELNOTES: None. PiperOrigin-RevId: 190957255
* Remove startup JVM version checkGravatar cushon2018-03-29
| | | | | | | | | | | | This was added during the JDK 7->8 transition to improve the diagnostic when an older-than-supported host_javabase was used. The version number handling doesn't work with JDK 9 (see [1]), and using Bazel binaries with a bundled host_javabase avoid the error entirely so the message is less important. [1] http://openjdk.java.net/jeps/223 PiperOrigin-RevId: 190944476
* Remove the --allow_configurable_attributes startup option.Gravatar ccalvarin2018-03-27
| | | | | | | It was removed from the java code 4 years ago, mentioning it causes the server to crash at startup. RELNOTES: None. PiperOrigin-RevId: 190636455
* Fold in warning and error output into client logging.Gravatar ccalvarin2018-03-23
| | | | | | | To replace blaze_util::die and blaze_util::pdie as well, FATAL statements need to accept blaze exit codes. RELNOTES: None. PiperOrigin-RevId: 190285798
* Update the client's skeleton logging framework to use it for --client_debug.Gravatar ccalvarin2018-03-21
| | | | | | | | | We are still unable to turn this on to write to files, but there are currently 2 logging systems in use in the client: the inactive one, and the print-to-stderr option triggered by --client_debug. Combine these, so we can use the same logging format for both. Also combine it with the VerboseLogging functionality - it was not documented anywhere and seems redundant. RELNOTES: None. PiperOrigin-RevId: 189979051
* blaze.cc: Remove unused had_to_wait variableGravatar David Ostrovsky2018-03-21
| | | | | | Closes #4851. PiperOrigin-RevId: 189897065
* Prefix startup dots with "Connecting to local Bazel server..." message.Gravatar ruperts2018-03-12
| | | | | RELNOTES: None. PiperOrigin-RevId: 188744724
* Expose --output_user_base to the Bazel server processGravatar Klaus Aehlig2018-02-27
| | | | | | | | | | | ...so that it can use that path to compute other directories in the output user base, in particular the default location for caches. The first cache we will add is the hash-index cache for downloads of external archives, but a spawn cache might be added later in the output user base as well. Change-Id: I24b1c33235c8f76ec008ecb1789163de2b2a45be PiperOrigin-RevId: 187164275
* Add a "direct" mode to "blaze run" that makes the process being run a directGravatar lberki2018-02-05
| | | | | | | | | | | | | | child of the process where the Blaze client itself was run. Limitations: - Untested on Windows; it should work because ExecuteProgram() is implemented there, too, but since Windows doesn't support exec(), there is at least one process in between Progress towards #2815. RELNOTES[NEW]: The new "--direct_run" flag on "blaze run" lets one run interactive binaries. PiperOrigin-RevId: 184528845
* Use the workspace name in the process title of the Bazel server.Gravatar philwo2018-02-01
| | | | | | | | This makes way more sense than using the name of the *parent* directory of the workspace. Closes #4253 - thanks @akira-baruah for suggesting this change and making it happen! PiperOrigin-RevId: 184147456
* Remove unnecessary debugging code.Gravatar philwo2018-01-31
| | | | PiperOrigin-RevId: 183995676
* Bazel client: address very subtle race condition during self extraction.Gravatar nharmata2018-01-30
| | | | | | | | | Fail fast if we are about extract the Bazel binary for version V into the predetermined install_base directory for version V'. This fixes Bazel's weakness to a race condition that puts it in a stable inconsistent state where all current and future attempts to use Bazel at version V will actually use V'. A rerun of the Bazel client will be able to successfully use version V'. This race condition occurs when the Bazel binary is replaced after determine the install directory (via the install_base_key file in the Bazel zip) but before we extract the files in the zip into the directory. RELNOTES: None PiperOrigin-RevId: 183843099
* Support explicitly specifying a location for jvm.outGravatar michajlo2018-01-29
| | | | | | | | Allows users to monitor server output without needing to fish the output base. Windows support is copied more or less verbatim from recommendations, I unfortunately don't know how to test this on windows. PiperOrigin-RevId: 183674130
* Emit Blaze server pid when there is a connection timeout.Gravatar felly2017-11-30
| | | | | RELNOTES: None PiperOrigin-RevId: 177420511
* Change config expansion application order, gated by startup flag ↵Gravatar ccalvarin2017-11-20
| | | | | | | | | | | | | --expand_configs_in_place. --config options were expanded in a fix-point expansion, where in practice, the flags that --config values expanded to ended up between the normal bazelrc options and the command line's explicit options. Since the options parser has an order-based priority scheme and it accepts multiple mentions of a single-valued option, this conflicts with users' expectations of being able to override these config expansions by using the order in which they are mentioned. This change makes it possible to expand the config values defined in your bazelrc (or blazerc) files to occur in-place: --stuff --config=something --laterstuff will interpret the options that --config=something expands to as if they had been mentioned explicitly between --stuff and --laterstuff. In order to not break users relying on complex flag combinations to configure their builds, this behavior will not yet be turned on by default. Instead, use --expand_configs_in_place as a startup flag to test this feature. --announce_rc may be helpful for debugging any differences between the fixed point and in-place expansions. Once you've debugged your problems, add "startup --expand_configs_in_place" to your blazerc to stick to the new behavior. RELNOTES: Use --expand_configs_in_place as a startup argument to change the order in which --config expansions are interpreted. PiperOrigin-RevId: 176371289
* Some minor quality of life improvements related to the fact that the default ↵Gravatar nharmata2017-11-07
| | | | | | | | | | value of --max_idle_secs is 15s when the TEST_TMPDIR environment variable is set. (i) Add a log line to blaze.INFO when the server shuts itself down due to idleness. (ii) Mention the --max_idle_secs default in the existing stderr spam when TEST_TMPDIR is set. RELNOTES: None PiperOrigin-RevId: 174745511
* Make the error message emitted when the server crashes during startup a bit ↵Gravatar lberki2017-10-26
| | | | | | | more intelligible. RELNOTES: None. PiperOrigin-RevId: 173512953
* client: deduplicate classpath for Bazel serverGravatar Laszlo Csomor2017-10-23
| | | | | | | Fixes https://github.com/bazelbuild/bazel/issues/3938 Change-Id: Ic837f1b97cf8469434118f2660fb15d14c035a14 PiperOrigin-RevId: 173100466
* Automated rollback of commit 61977f4ae57c877fc28be176f36c28642b276eec.Gravatar lberki2017-10-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks a test of Bazel on Windows: http://ci.bazel.io/view/Dashboard/job/bazel-tests/1078/testReport/src_test_py_bazel_bazel_clean_test/exe/src_test_py_bazel_bazel_clean_test_exe/ Fixes #3882. *** Original change description *** Don't release the client lock while the server executes a command. The server still doesn't support concurrency, even for commands like 'help', so there's no benefit from releasing it. This also improves progress messages. Today the client may print WARNING: Running Blaze server needs to be killed, because the startup options are different. and then wait indefinitely while the server finishes processing a request. With this change, an explanation for the delay is given. RELNOTES: None. GO... *** RELNOTES: None. PiperOrigin-RevId: 171689429
* Don't release the client lock while the server executes a command. TheGravatar Googler2017-10-10
| | | | | | | | | | | | | | | server still doesn't support concurrency, even for commands like 'help', so there's no benefit from releasing it. This also improves progress messages. Today the client may print WARNING: Running Blaze server needs to be killed, because the startup options are different. and then wait indefinitely while the server finishes processing a request. With this change, an explanation for the delay is given. RELNOTES: None. PiperOrigin-RevId: 171571670
* Internal changeGravatar Googler2017-08-30
| | | | PiperOrigin-RevId: 166874758
* Bazel client: debug info for corrupt installationGravatar László Csomor2017-08-28
| | | | | | | | | | | | | | | | | | | | The Bazel client checks the installation files every time it starts. Bazel checks that it can read the installation files and that their mtimes are in the future. The "bazel-tests" project on Bazel CI recently produced flaky-looking failures. The error messages talk about a missing installation file. This change makes Bazel print more debugging info when the aformentioned check fails. I hope the information will help catching this bug. See https://github.com/bazelbuild/bazel/issues/3618 Change-Id: Ic51384b5840262484515fa320f162a79cfd0e012 PiperOrigin-RevId: 166696141
* Propagate detected value of BAZEL_SH to --client_env.Gravatar Dmitry Lomov2017-08-22
| | | | | | | | | | | | Also adds some extra debug logging. Also removes some msys-only code paths. Fixes #3498. This is a reland of commit 2f38404. Change-Id: I1690a64f427070cc3f127b857650e6a32d1aab35 PiperOrigin-RevId: 166049222
* Automated rollback of commit 2f38404728a12840e416566b453e9cdebab5d859.Gravatar elenairina2017-08-18
| | | | | | | | | | | | | | | | | | | | | Fixes #3584 *** Reason for rollback *** Broke bazel_windows_test fails on windows. *** Original change description *** Propagate detected value of BAZEL_SH to --client_env. Also make debug_log available as soon as possible on startup. Also adds some extra debug logging. Also removes some msys-only code paths. Fixes #3498. Change-Id: I5b769f2c5a728106e5252869745ec79e555cbaf2 PiperOrigin-RevId: 165692468
* Propagate detected value of BAZEL_SH to --client_env.Gravatar Dmitry Lomov2017-08-18
| | | | | | | | | | | Also make debug_log available as soon as possible on startup. Also adds some extra debug logging. Also removes some msys-only code paths. Fixes #3498. Change-Id: I5b769f2c5a728106e5252869745ec79e555cbaf2 PiperOrigin-RevId: 165574022
* Send Bazel startup options to server.Gravatar ccalvarin2017-08-16
| | | | | | | | | | | Send the startup options tagged with their origin so that the server has correct information about the command line as the client received it. Removes the unconditional stderr printing of all bazelrc startup options in the bazel client. Instead, the startup options are sent to the server and the same informational printing is gated on the --announce_rc option. This avoids unconditional log spam to stderr early in startup. If the server is unreachable or there are errors parsing startup options, the message is still printed to stderr. Fixes https://github.com/bazelbuild/bazel/issues/2530. RELNOTES: --announce_rc now controls whether bazelrc startup options are printed to stderr. PiperOrigin-RevId: 165211007
* Windows: detect if started from Windows ExplorerGravatar Laszlo Csomor2017-07-28
| | | | | | | | | | | | | Bazel now detects whether it was started from a command line (or as a subprocess), or from Windows Explorer (by clicking its icon). In the latter case it displays an error message asking the user to run it from a command line, and waiting for a keypress so the user has time to read this message. Change-Id: I4b0430e30d2f1f243cec6ff63cb3abac907e60e3 PiperOrigin-RevId: 163338527
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-30 06:27:19 +0000