aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/cpp/blaze_util_posix.cc
Commit message (Collapse)AuthorAge
* Fix block_for_lock.Gravatar ccalvarin2018-07-23
| | | | | | | | | | | | | | | | | | | | | | | | | | On two fronts: First, it should follow standard command line semantics. Second, it should work as intended: --noblock_for_lock means the client will not wait for another command to finish, but will exit eagerly. It can be useful for preventing hanging in applications that are non-interactively calling bazel. It should have standard startup-option semantics: the default value is accepted as a no-op or can be provided to override a previous value. The next issue involves 2 different locks - the client lock, and the server-side command lock. This duality exists because we would like, one day, to be able to run certain commands, like info or help, at the same time, so multiple commands would need specialized locks that allow some duality but blocks others. This can only be done at the server level, so as soon as the client gets the "we're connected" grpc message from the server, it releases the client lock and lets the server manage multiple requests. There are basically 3 possible states that are relevant to this option: 1) no other client is active, so no one holds the client lock or the command lock - the server can be used, shutdown or started as needed. - no blocking, but no need to block, either, so we're safe 2) another client (client1) holds the lock, but it is currently using a server that we want to reuse. If client1 still holds the client lock, we fail fast. Same thing if client1 is holding the server-side lock: we will exit gracefully when the BlazeCommandDispatcher responds with a failure. 3) client1 holds the lock but its server cannot be reused. (batch clients also fall into this category, as there is no server to reuse - but in this case, the client lock is still in play). However, for server mode, this is broken - the following happens: - Server is occupied with client1's request, holds the command lock - client2 wants to restart the server, so sends the old server a "shutdown" command - the BlazeCommandDispatcher says - nuh-uh, this is busy, and you said you didn't want to wait for the lock - client2 absorbs this response - waits (blocks...) - for a minute - then force shuts-down the old server. So we had 2 problems - we block, and we shutdown a server that we truly intended to keep going. Now, if the server responds saying another action is using it, the client will exit correctly, and leave the old server to do its thing. RELNOTES: None. PiperOrigin-RevId: 205671817
* Windows,Bazel client: multithreaded file writingGravatar Laszlo Csomor2018-07-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Bazel client on Windows now writes extracted binaries to disk in parallel. On all other systems it writes them serially (as before). This change makes blaze.cc:ActuallyExtractData() about 3x faster when using a HDD. (In previous experiments I saw no speedup with multi-threaded writing on machines with an SSD.) The Windows-specific code uses the native Threadpool API of Windows, creating a pool of at least 8 and at most 16 threads. (This seems to be a good balance between speed and thread count.) The OS manages everything about the pool; Bazel submits callbacks and the pool executes them asynchronously. blaze.cc:ActuallyExtractData() speed, before: - Windows: 6.48s (avg) / 6.38s (median) - Linux (Debian): 4.78s (avg) / 4.79s (median) blaze.cc:ActuallyExtractData() speed, after: - Windows (8-16 threads): 2.05s (avg) / 2.01s (md) - Windows (1 thread): 5.77s (avg) / 5.74s (median) See https://github.com/bazelbuild/bazel/issues/5444 Change-Id: I7211f3d28eb8b9837352c16ff8df0411d5a9ebe1 Closes #5600. Change-Id: I7a74d62a563c92948a4dfa8ad5ac83eae018db10 PiperOrigin-RevId: 204891217
* blaze_util_posix.cc: fix order of #defineGravatar Klaus Aehlig2018-07-05
| | | | | | | | | | On some systems (including some versions of FreeBSD) it is a requirement that _WITH_DPRINTF be defined for <stdio.h> to provide the appropriate header for dprintf(3). Therefore, the respective #define has to come before any #include as those might pull is <stdio.h>. Change-Id: I25d55c9c7c0912e8619faf774d2e09f9af9a6a53 PiperOrigin-RevId: 203351202
* Make the system bazelrc configurable.Gravatar ccalvarin2018-06-20
| | | | | | | | | | | The master bazelrc is now defined by preprocessor macro at (Bazel's) compile time. The default is still /etc/bazel.bazelrc for most platforms, but windows now has a %ProgramData% relative default value as well. Users wishing to change this default when building Bazel for a new platform should edit BAZEL_SYSTEM_BAZELRC_PATH in src/main/cpp/BUILD. Part of https://github.com/bazelbuild/bazel/issues/4502, relevant to the duplicate issue #4809. TESTED: default settings were tested manually, since they cannot be tested in a sandbox RELNOTES: Windows default system bazelrc is read from the user's ProgramData if present. PiperOrigin-RevId: 201423446
* Move path-manipulation functions to own library file.Gravatar ccalvarin2018-06-05
| | | | | | | | | | | Leave functions that make file accesses in the file library, and general blaze utilities in the blaze_util file, but move the functions that boil down to string manipulation and path formatting to their own file. (With the exception of getCWD, since absolute path syntax is relevant here.) Doing this largely to consolidate all Windows path control into a single place, so that it's easier to notice inconsistencies. For instance, ConvertPath currently makes Windows paths absolute, but not Posix paths, and MakeAbsolute relies on this behavior. In addition, JoinPath assumes Posix path syntax, which leads to some odd looking paths. These will be fixed in a followup change. (Found these issues while working on #4502, trying to fix the windows-specific system bazelrc.) RELNOTES: None. PiperOrigin-RevId: 199368226
* Keep LANG/LANGUAGE and related environment variables for the process spawned ↵Gravatar lberki2018-04-18
| | | | | | | by "blaze run --direct_run". RELNOTES: None. PiperOrigin-RevId: 193391379
* Delete unused RunProgram function.Gravatar Benjamin Peterson2018-04-10
| | | | | | | Closes #4986. Change-Id: I81bbec801116ec1f45416bba7a724d7f00b9b00c PiperOrigin-RevId: 192253687
* Remove die() and replace it with BAZEL_DIE, part of the logging framework.Gravatar ccalvarin2018-04-06
| | | | | | | This will mean the messages will make it to the right output stream. RELNOTES: PiperOrigin-RevId: 191925662
* Remove pdie.Gravatar ccalvarin2018-03-29
| | | | | | | pdie and die are pretty similar, pdie just adds the errno string or equivalent from GetLastErrorString(). Make this explicit. This makes message formatting more clear in preparation for moving these all to BAZEL_LOG. RELNOTES: None. PiperOrigin-RevId: 190957255
* Remove startup JVM version checkGravatar cushon2018-03-29
| | | | | | | | | | | | This was added during the JDK 7->8 transition to improve the diagnostic when an older-than-supported host_javabase was used. The version number handling doesn't work with JDK 9 (see [1]), and using Bazel binaries with a bundled host_javabase avoid the error entirely so the message is less important. [1] http://openjdk.java.net/jeps/223 PiperOrigin-RevId: 190944476
* Update the client's skeleton logging framework to use it for --client_debug.Gravatar ccalvarin2018-03-21
| | | | | | | | | We are still unable to turn this on to write to files, but there are currently 2 logging systems in use in the client: the inactive one, and the print-to-stderr option triggered by --client_debug. Combine these, so we can use the same logging format for both. Also combine it with the VerboseLogging functionality - it was not documented anywhere and seems redundant. RELNOTES: None. PiperOrigin-RevId: 189979051
* Support explicitly specifying a location for jvm.outGravatar michajlo2018-01-29
| | | | | | | | Allows users to monitor server output without needing to fish the output base. Windows support is copied more or less verbatim from recommendations, I unfortunately don't know how to test this on windows. PiperOrigin-RevId: 183674130
* Remove dead ConvertPathListGravatar Loo Rong Jie2018-01-26
| | | | | | Closes #4494. PiperOrigin-RevId: 183380779
* Initialize server_pid in an attempt to convince a linter that ↵Gravatar lberki2018-01-09
| | | | | | | | | ExecuteDaemon() always returns a defined value. Fixes #4394. RELNOTES: None. PiperOrigin-RevId: 181315375
* Add return statements to silence compiler warning.Gravatar John Cater2017-12-06
| | | | | Change-Id: Ic39712b613a0400f595a7db6e6f957057775b1b8 PiperOrigin-RevId: 178101141
* Emit Blaze server pid when there is a connection timeout.Gravatar felly2017-11-30
| | | | | RELNOTES: None PiperOrigin-RevId: 177420511
* Print 'waiting for other blaze command' on its own line.Gravatar Googler2017-11-28
| | | | | | | | | | It now prints a newline char immediately, rather than waiting until the lock is released, and printing 'done!' at the end of the line. This allows per-line parsers (like IntelliJ) to display this progress notification without special-case hacks. PiperOrigin-RevId: 177180918
* blaze_util_posix.cc: declare use of dprintfGravatar Klaus Aehlig2017-10-09
| | | | | | | | | | On some systems, the dprintf header is only part of <stdio.h> if requested (to allow compatibility with applications before the standardization of dprintf). As we use dprintf, request it by defining _WITH_DPRINTF. Change-Id: Ia4a80e08760043c5343b4c333b146f4ea87d081b PiperOrigin-RevId: 171409630
* In the launcher, use F_OFD_SETLK if available.Gravatar Googler2017-10-06
| | | | | | | | | | | Because OFD locks are not per-process, F_OFD_GETLK does not report which process holds the lock, so we use the contents of the lock file instead. Works around a Linux kernel bug where POSIX record locks are lost at execve() if any other threads exist. RELNOTES: None. PiperOrigin-RevId: 171209685
* Windows: Adding default --python_pathGravatar Yun Peng2017-09-14
| | | | | | | | | | | On Windows, Bazel client will try to find python.exe in $PATH. If succeed, then we pretend to add a --python_path option in the least important bazelrc file. Fixed https://github.com/bazelbuild/bazel/issues/3717 Change-Id: I8d97b0895f024d8d236f3b4b39f91c41d947a5fa PiperOrigin-RevId: 168659085
* Ensure our "Another command is running" messages hold realistic PIDs.Gravatar jmmv2017-08-28
| | | | | | | | | | | | | | | | | | | | Instead of blocking indefinitely for the server lock to become available when other commands are running, busy-wait for the lock to be released. This allows us to detect changes in the PID of the client that is holding the lock, and thus lets us make our wait messages more accurate. There have been multiple bug reports over time (especially from macOS users) where they complain that Bazel is stuck waiting for a non-existent PID. The code doesn't look obviously bogus, so this might just be a case of confusion based on the printed PID. By improving the diagnostic messages we output, we'll either make this confusion go away or have a chance of gathering more data when/if this happens again. This change has the side-effect of homogenizing the wait messages printed by both the Blaze client and the Blaze server and also adds details to know which component is printing what. PiperOrigin-RevId: 166508406
* blaze_util_posix.cc: fix call to nanosleepGravatar Klaus Aehlig2017-08-09
| | | | | | | | | | | The struct timespec consists of a time_t and a long entry. So provide arguments of correct type, instead of two times an unsigned int, and explicitly cast appropriately. Fixes type errors on 32-bit machines. While there, also explicitly inlcude time.h, required for nanosleep. Change-Id: I56aabed1cc0c82d93f9c4b9b69d2c9d549207855 PiperOrigin-RevId: 164599502
* Windows: detect if started from Windows ExplorerGravatar Laszlo Csomor2017-07-28
| | | | | | | | | | | | | Bazel now detects whether it was started from a command line (or as a subprocess), or from Windows Explorer (by clicking its icon). In the latter case it displays an error message asking the user to run it from a command line, and waiting for a keypress so the user has time to read this message. Change-Id: I4b0430e30d2f1f243cec6ff63cb3abac907e60e3 PiperOrigin-RevId: 163338527
* Include <cinttypes> instead of <stdint.h>Gravatar Damien Martin-Guillerez2017-07-27
| | | | | | | | | | | cinttypes is the C++ header that should replace stdint.h. Not using the correct header was leading to compilation error on CentOS 6.7 Fixes #3455. To be cherry-picked for #3375. Change-Id: I6df22134a4a4902ec9fa7ecdfaeb5408eacf3564 PiperOrigin-RevId: 163334651
* Implement bash.exe detection logic.Gravatar Dmitry Lomov2017-07-19
| | | | | | | | | | | | | | | | The logic is as follows: 1) search for msys installation 2) search got git-on-Windows installation 3) search in PATH. This happens on every client startup unless BAZEL_SH enviornment variable is set. My measurements show that the time required for this detection is negligible (<10 msec in the worst case). Change-Id: If130e2491a9df5a23954d303f2ccdb932eeed1db PiperOrigin-RevId: 162466913
* Ensure that shutdown commands end the server process before completionGravatar mschaller2017-07-12
| | | | | | | | | | | | | | | | | | | | This change ensures that the server process is terminated before the client process terminates, when evaluating a command that shuts down the server. When completing such a command, the server communicates to the client that the server will terminate itself by setting a termination_expected bit in the final RunResponse message. The client then waits up to 60s for the server process to actually terminate. If it does not, then the client SIGKILLs the server. Also makes the gRPC server stop accepting new commands before the shutdown command completes. Drive-by fix to comments on Search{Un,Null}aryOption. RELNOTES: Commands that shut down the server (like "shutdown") now ensure that the server process has terminated before the client process terminates. PiperOrigin-RevId: 161537480
* Automated rollback of commit 458990b0c155130e242117e2bfc5ebfdf787d2e2.Gravatar Googler2017-07-11
| | | | | | | | | *** Reason for rollback *** A backwards incompatible change for CLI. RELNOTES: Rollback of https://github.com/bazelbuild/bazel/commit/458990b0c155130e242117e2bfc5ebfdf787d2e2 PiperOrigin-RevId: 161457646
* Raise the maximum number of processes and open files to their maximum.Gravatar jmmv2017-07-11
| | | | | | | | | | | | | | | | | | | Under macOS, the default soft resource limits for open files and concurrent processes are pretty low, but their corresponding hard defaults are reasonable. Because the soft limits are low, Bazel sometimes fails during large builds -- not because of Bazel itself, but because the executed actions do "too much work" or because the --jobs setting was high enough to cause all parallel tasks to exceed the limits. Instead of trying to fix the actions themselves, start by trying to raise the system limits as a best-effort operation. And, given that this code is fairly portable, try to do it on all POSIX systems and not just macOS. Note that, for non-macOS systems, this might still not do what's promised in all circumstances because I'm currently only implementing GetExplicitSystemLimit on macOS. RELNOTES: None. PiperOrigin-RevId: 161401482
* Look at stderr for terminal detection.Gravatar Googler2017-07-10
| | | | | | | | | | | | | | | | | | | | bazel prints all the progress to stderr, yet the decision to switch between "smart" and "dumb" output modes is done based on the stdout and stderr connected terminals. That breaks for a command like this: bazel query 'something' | wc -l Even though all the progress is still output to the terminal through stderr, bazel switches to "dumb" mode, printing progress messages one per line. It seems reasonable to make the "smart"/"dumb" output mode decision based on the stderr only. Tested: With the change "bazel query '...' | wc -l" prints "smart" progress messages. RELNOTES: Check stderr to detect if connected to a terminal. Deprecate --isatty. PiperOrigin-RevId: 161243017
* fix signedness warnings in blaze_util_posix.ccGravatar Benjamin Peterson2017-05-31
| | | | | | | | | | if (result != count) { ~~~~~~~^~~~~~~~ Of course, problems arising from this are only theoretical. Change-Id: Id045a97623026e005d40940eb4a7d30d73f4ce32 PiperOrigin-RevId: 157561218
* Fix a typo and remove an unnecessary scary warning submitted in unknown commit.Gravatar lberki2017-05-15
| | | | | RELNOTES: None. PiperOrigin-RevId: 156039435
* Remove all complex logic after forking.Gravatar lberki2017-05-11
| | | | | | | | | | This is so that there is as little room for undesired behavior (e.g. deadlocks), which can happen when multi-threaded programs fork if the code is not written *very* carefully. See the comment before ExecuteDaemon() in blaze_util_posix.cc for an detailed treatise on the matter. RELNOTES: None. PiperOrigin-RevId: 155750707
* Update ExecuteDaemon() on POSIX systems so that the client writes the PID ↵Gravatar lberki2017-05-10
| | | | | | | | | file and not the server. This is so that the server does as few things as possible before exec() (preferably, nothing) so that we don't accidentally call malloc() which would make it possible to deadlock if the server spawned multiple threads before ExecuteDaemon(). RELNOTES: None. PiperOrigin-RevId: 155603273
* cpp: turn ListSeparator() function into a constantGravatar Thiago Farina2017-04-25
| | | | | | | | It does not need to be a fully allocated std::string every time. A simple character constant is enough for it. Change-Id: I98b9d4bb77932ea18646fbc793132e089bc66124 PiperOrigin-RevId: 154054987
* Bazel client: simplify ReadDirectorySymlinkGravatar laszlocsomor2017-03-31
| | | | | | | | | | | | | | | This is a bugfixed version of https://cr.bazel.build/9520 which broke some Google-internal tests. This change allows removing duplicate code: ReadDirectorySymlink was implemented on Windows as a junction resolver, which is also implemented in file_windows.cc. Now it uses the JunctionResolver. RELNOTES: none PiperOrigin-RevId: 151691895
* Bazel client, Windows: implement GetHomeDir Gravatar László Csomor2017-03-23
| | | | | | | | | | | | | Create a method in blaze_util_<platform> to retrieve the path to the home dir ($HOME on Linux/macOS, %USERPROFILE% on Windows), where we look for the user's bazelrc file (".bazelrc"). -- Change-Id: I86be1dbe1f992ad55eb09b496024754099d54912 Reviewed-on: https://cr.bazel.build/9513 PiperOrigin-RevId: 151004759 MOS_MIGRATED_REVID=151004759
* Catch attempts to fork from a multithreaded bazel launcher process.Gravatar Googler2017-03-16
| | | | | | | | This is prone to deadlock. -- PiperOrigin-RevId: 150263255 MOS_MIGRATED_REVID=150263255
* Bazel client: simplify {Read,Write}File semanticsGravatar Laszlo Csomor2017-03-01
| | | | | | | | | | | | | | | | Introduce a platform-specific file handle type (HANDLE on Windows, int on Linux/Darwin/FreeBSD) so we can get rid of the read_func and write_func functions, since they are always the same everywhere. Also include file_platform.h in file.h, since they are logically the same file (file_platform.h is just the platform-specific part of file.h). -- PiperOrigin-RevId: 148892736 MOS_MIGRATED_REVID=148892736
* Bazel client: make jvm.log path platform-safe Gravatar László Csomor2017-02-24
| | | | | | | | | | | | | | | | | When writing the javalog.properties file that configures the java logger, pass the log file path in a platform safe manner. Namely if we're running on Windows, don't write paths with backslashes because they are mistaken for paths with escaped characters. Fixes: https://github.com/bazelbuild/bazel/issues/2576 -- Change-Id: Ibd907c13f1ffe4561c3a4e737f0c2b25ec5d4b17 Reviewed-on: https://cr.bazel.build/9059 PiperOrigin-RevId: 148342757 MOS_MIGRATED_REVID=148342757
* Bazel client: split CanAccess to specific methodsGravatar Laszlo Csomor2017-01-11
| | | | | | | | | | | | | | The new methods (CanReadFile, CanExecuteFile, CanAccessDirectory) are a lot easier to implement on Windows than a generic CanAccess. On POSIX these methods are just a wrapper around the now static-visible CanAccess(). See https://github.com/bazelbuild/bazel/issues/2107 -- PiperOrigin-RevId: 144176710 MOS_MIGRATED_REVID=144176710
* Bazel client: add comments, delete dead codeGravatar Laszlo Csomor2016-12-22
| | | | | | | | | | In this change: - add TODOs and comments - inline blaze::ExitImmediately at call sites -- PiperOrigin-RevId: 142671757 MOS_MIGRATED_REVID=142671757
* Bazel client: generalize path handlingGravatar Laszlo Csomor2016-12-19
| | | | | | | | | | | | | | Use/implement utility methods to join paths, check if they are the root directory or are absolute, etc. Doing so (instead of say checking if a path starts with "/") allows for correct behavior on Windows. See https://github.com/bazelbuild/bazel/issues/2107 -- PiperOrigin-RevId: 142446027 MOS_MIGRATED_REVID=142446027
* Bazel client: get rid of RunProgramGravatar Laszlo Csomor2016-12-15
| | | | | | | | | | | | | | | | | | Move GetJvmVersion from blaze_util to blaze_util_platform, and remove the RunProgram declaration from blaze_util.h. Since GetJvmVersion is the only user of RunProgram this is safe to do, and allows making RunProgram static as well as simplifying its implementation on Windows, while also changing it to use CreateProcessW instead of CreateProcessA. See https://github.com/bazelbuild/bazel/issues/2181 -- PiperOrigin-RevId: 142122045 MOS_MIGRATED_REVID=142122045
* move MakeDirectories() into file_platform.hGravatar Thiago Farina2016-12-07
| | | | | | | | | | | | | | This way we can remove the dependency on blaze_util (which is Bazel's client specific) from singlejar. This work was made possible by commit 49970e0136d0: ("Bazel client: platform-specific {Read,Write}File"). -- Change-Id: I6e95fb9119a271e4d48cbe2eefc1d5354ee188aa Reviewed-on: https://cr.bazel.build/7650 PiperOrigin-RevId: 141294165 MOS_MIGRATED_REVID=141294165
* Bazel client: make it compile with MSVCGravatar Laszlo Csomor2016-11-28
| | | | | | | | | | | | | | | | | | | | | | | | | Final modifications to the Bazel client code so we can compile //src/main/cpp/...:all using MSVC. Yay! We still have some dependencies that don't compile with MSVC, namely Ijar, build-runfiles, process-wrapper, and process-tools. Still, this change is a huge success, because now we can add regression tests to prevent people from introducing breaking changes to the client that would break Windows/MSVC compilation. It's important to point out that we can only build this library for now, most functions in file_windows.cc and blaze_util_windows.cc have an empty body (they just call `pdie`). See https://github.com/bazelbuild/bazel/issues/2107 -- MOS_MIGRATED_REVID=140348351
* Bazel client: more platform-specific logicGravatar Laszlo Csomor2016-11-28
| | | | | | | | | | Move terminal-querying functions and GetUserName from blaze_util.cc into blaze_util_<platform>. See https://github.com/bazelbuild/bazel/issues/2107 -- MOS_MIGRATED_REVID=140346402
* Bazel client: platform-specific {Read,Write}FileGravatar Laszlo Csomor2016-11-28
| | | | | | | | | | | | | | | | Move blaze::ReadFile and blaze::WriteFile to file.h and file_platform.h (thus into the blaze_util namespace), and update references. This allows us to implement these methods in a platform-specific way. Also move UnlinkPath. See https://github.com/bazelbuild/bazel/issues/2107 -- MOS_MIGRATED_REVID=140328273
* Bazel client: platform-dependent lock handlingGravatar Laszlo Csomor2016-11-28
| | | | | | | | | | Move blaze::AcquireLock and blaze::ReleaseLock into blaze_util_<platform>. See https://github.com/bazelbuild/bazel/issues/2107 -- MOS_MIGRATED_REVID=140200355
* Bazel client: mock out read/write callsGravatar Laszlo Csomor2016-11-28
| | | | | | | | | | | | | | | | | | | | | | | | | Make blaze::ReadFileDescriptor(int fd, ...) and blaze::WriteFile(int fd, ...) platform-independent by mocking out the read(2) and write(2) calls. Also rename ReadFileDescriptor to ReadFrom and introduce a new WriteTo method that encapsulates WriteFile's prior logic. In particular these functions now take a read_func/write_func function argument instead of a file descriptor, so the read(2)/write(2) calls can be mocked out. This allows us to use these functions on Windows too, where read(2)/write(2) are not implemented, and we can inject a different read_func/write_func. See https://github.com/bazelbuild/bazel/issues/2107 -- MOS_MIGRATED_REVID=140195973
* Bazel client: platform-dependent signal handlingGravatar Laszlo Csomor2016-11-24
| | | | | | | | | | Move the signal handling code from blaze.cc into blaze_util_<platform>. See https://github.com/bazelbuild/bazel/issues/2107 -- MOS_MIGRATED_REVID=140134781
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-26 19:14:36 +0000