aboutsummaryrefslogtreecommitdiffhomepage
Commit message (Collapse)AuthorAge
* Prepare CROSSTOOL for dSYM simplificationGravatar Googler2018-07-10
| | | | | | | | Currently dSYM information is propagated using a zip file emitted by the linking action which contains a dwarf file and plist. A second action then unzips said file, discards the plist and propagates the dwarf file. This CL is part of a change which makes the link action emit the dwarf file directly. This change adds logic to the CROSSTOOL and wrapped_clang to temporarily support both the old and new approaches using the presence of DSYM_HINT_DSYM_BUNDLE_ZIP as a signal as to which to use. After this change the new (and temporary) CROSSTOOL feature no_dsym_create_zip will allow suppressing DSYM_HINT_DSYM_BUNDLE_ZIP from being passed. Once the migration is complete we will remove both the signal variable and feature. PiperOrigin-RevId: 203954795
* Remove ToolchainContextException and make exception handling more robust.Gravatar John Cater2018-07-10
| | | | | Change-Id: Iefeab0de85541b963c6cc262339e73b79f630f8b PiperOrigin-RevId: 203952823
* When using find_cpp_toolchain, make sure the relevant attribute, ↵Gravatar juliexxia2018-07-10
| | | | | | | '_cc_toolchain' exists on the ctx being passed to this function. (I broke TAP once because I forgot to add the attribute. Perhaps not that helpful this late in the migration but doesn't hurt.) RELNOTES: None. PiperOrigin-RevId: 203947282
* Synchronize on process factory to inhibit ETXTBSYGravatar George Gensure2018-07-10
| | | | | | | | | | | | | | | | | | | | | | | | | | Refocus synchronization mechanism to cope with file descriptor set fork- induced races to more tightly constrain concurrent fork/exec pairs. This problem has been observed in bazel proper repeatedly, exhibiting as the iconic ETXTBSY - Text file busy in wide worker pool builds and tests. Evidence that this was discovered by @buchgr is in the comment and change to the embedded ExecutionService implementation, and the description of the race and the need for the synchronization was lifted from that scope to the JavaSubprocessFactory. This factory is a singleton and represents the gateway to all worker process execution, and serves as the correct lock primitive to ensure that file descriptor sets are not duplicated across forks, which gave rise to this issue. To test this, I demonstrated a reproducer presented at https://bugs.java.com/view_bug.do?bug_id=8068370 with 2.4% of invocations in that pathological case exhibiting the issue. With a functionally equivalent change - synchronizing around a processBuilder.start() call - as the only modification to the reproducer, no further failures of any kind were observed, over several hundred runs. Closes #5556. PiperOrigin-RevId: 203947224
* Document differences between version and version_fileGravatar katre2018-07-10
| | | | | | | | | Add to the documentation that version_file and release_file cannot be used to generate an output with the N-V-R.A naming scheme. Fixes #5522. Closes #5555. PiperOrigin-RevId: 203946762
* Windows,tests: port bazel_build_event_stream_testGravatar Laszlo Csomor2018-07-10
| | | | | | | | | | | | //src/test/shell/bazel:bazel_build_event_stream_test now runs on Windows. See https://github.com/bazelbuild/bazel/issues/4292 Closes #5560. Change-Id: Ica3d59fb6f557c85d2fd646cf005ffd831046119 PiperOrigin-RevId: 203945734
* Remove old implementation for object file pathGravatar pcloudy2018-07-10
| | | | | | RELNOTES: --experimental_shortened_obj_file_path is removed. PiperOrigin-RevId: 203944948
* Track additionallyPrunableIncludes separately from declaredIncludeSrcs (don'tGravatar Googler2018-07-10
| | | | | | | | | | | | | | | | | | compose a nested set encompassing both). The current prunableHeaders were used in various places, at least two of which let to a duplicate iteration through the comparatively large declaredIncludeSrcs: CppCompileAction.computeKey() and CppCompileAction.getAllowedDerivedInputs(). Also do some other minor optimizations: - CcCompileActionBuilder.buildAllInputs() now just returns a NestedSet with the same order as its parameter. As inputsForValidation is almost always empty, this makes this NestedSet pass the underlying one through. - CcCompilationContext.directModuleMaps is just a list of the module maps of direct dependencies. As such, there is no use for keeping them in a NestedSet. RELNOTES: None. PiperOrigin-RevId: 203938011
* Bazel server, VFS: revert to using asByteSourceGravatar laszlocsomor2018-07-10
| | | | | | | | | | | | | | | | | | commit 59f17d6e0550bf63a0b6ef182e2d63474e058ede updated files to use try-with-resources when dealing with streams. The change also got rid of asByteSource, replacing it with throw-away ByteSource instances wrapping a try-with-resources-guarded InputStream. Doing so was unnecessary though, because all ByteSource methods (except for open[Buffered]InputStream) close the stream. Thanks to @jbduncan to point that out and explain in detail [1]. [1] see comment thread on https://github.com/bazelbuild/bazel/commit/59f17d6e0550bf63a0b6ef182e2d63474e058ede under `FilesetManifest.java` RELNOTES: none PiperOrigin-RevId: 203934830
* Only compile a single source file once for a given cc_* rule.Gravatar lberki2018-07-10
| | | | | RELNOTES: None. PiperOrigin-RevId: 203934582
* Bazel server, tools: ensure Writers are closedGravatar laszlocsomor2018-07-10
| | | | | | | | | | | | | | | | | | | | Follow-up to commit 09d20311d982606093ed881d779bb05a5ee70ed3. Use try-with-resources to ensure Writer objects are closed eagerly. Eagerly closing Writers avoids hanging on to file handles until the garbage collector finalizes the object, meaning Bazel on Windows (and other processes) can delete or mutate these files. Hopefully this avoids intermittent file deletion errors that sometimes occur on Windows. See https://github.com/bazelbuild/bazel/issues/5512 RELNOTES: none PiperOrigin-RevId: 203934471
* Use generic classes instead of cquery specific class.Gravatar twerth2018-07-10
| | | | | | | Also introduce NamedThreadSafeOutputFormatterCallback for common code. RELNOTES: None PiperOrigin-RevId: 203932877
* pkg: fix documentation (docker -> archive)Gravatar Klaus Aehlig2018-07-10
| | | | | | | pkg_tar creates an archive, not a docker image. Change-Id: I402bd8fdb67c38cd67a3990b983aa3b6d912da37 PiperOrigin-RevId: 203932291
* buildkite: re-enable determinism test on macOSGravatar buchgr2018-07-10
| | | | | RELNOTES: None PiperOrigin-RevId: 203931666
* Update third_party/protobuf to 3.6.0Gravatar Loo Rong Jie2018-07-10
| | | | | | Fixes #5439. PiperOrigin-RevId: 203917717
* third_party/protobuf: update protobuf in third_party to version 3.6.0Gravatar Jakob Buchgraber2018-07-10
| | | | Split out from https://github.com/bazelbuild/bazel/pull/5439
* Restructure the NestedSet usage in CcCompilationContext. Goals:Gravatar Googler2018-07-10
| | | | | | | | | | | | | | | | | | - Don't use NestedSets of NestedSets. Not sure whether this is a performance improvement or has serialization benefits, but it does make understanding the structure a lot easier. - Use the same large NestedSet for all header information (whether or not they are modular and what pregrepped headers might exist). This speeds up the include scanner calls getLegalGeneratedScannerFileMap and getModularHeaders because they now iterate over just a single NestedSet and the second evaluation of a NestedSet is much cheaper. In a subsequent change, we can likely fold all three iterations over this NestedSet into one (in getLegalGeneratedScannerFileMap, getModularHeaders and getUsedModules). Measurements show a small reduction in heap usage. RELNOTES: None. PiperOrigin-RevId: 203909308
* third_party/googleapis: sync error_details.protoGravatar Jakob Buchgraber2018-07-10
|
* Set INCLUDE and LIB env var for actions that actually need themGravatar Loo Rong Jie2018-07-10
| | | | | | | | | | This makes console output of `bazel build -s` shorter when debugging `CROSSTOOL`. /cc @meteorcloudy Closes #5539. PiperOrigin-RevId: 203899866
* Automatic code cleanup.Gravatar gregce2018-07-09
| | | | PiperOrigin-RevId: 203841642
* Pull up calls to ToolchainContext.resolvedToolchainLabels and finishGravatar John Cater2018-07-09
| | | | | | | ToolchainContext configuration earlier. Change-Id: I034f56a8a181aac19ecb38e51558ccdc6c0c7a6a PiperOrigin-RevId: 203835978
* Fix CcCompilationInfo to appropriately subclass Struct, and improve upon ↵Gravatar cparsons2018-07-09
| | | | | | | error messaging and commenting when there's an unresolvable skylark type. RELNOTES: None. PiperOrigin-RevId: 203826504
* Tell user to blaze clean --expunge.Gravatar dmaclach2018-07-09
| | | | PiperOrigin-RevId: 203813666
* Fix typo in javadoc for FileValue#realRootedPath.Gravatar shreyax2018-07-09
| | | | PiperOrigin-RevId: 203813658
* [Reland] Accept proto paths relative to proto_source_root as direct ↵Gravatar Googler2018-07-09
| | | | | | | | | | | | | | | dependencies. This is a reland of https://github.com/bazelbuild/bazel/commit/5deca4cf88f5568771f2c836a9b8c693b88bd749. This will make protoc see as direct dependencies the .proto files that were included using the proto_source_root flag. Until now, Bazel passed to protoc the direct dependencies of a target as the path relative to the WORKSPACE, which made it fail when a shorter path, relative to the package was used. Progress on #4544. RELNOTES: None. PiperOrigin-RevId: 203808292
* openjdk: update macOS openjdk image. Fixes #5532Gravatar buchgr2018-07-09
| | | | | | | | | | | I missed the java.instrument module when building the smaller openjdk. I have updated the jdk to include this module. This was the cause for #5532. The image was created by https://buildkite.com/bazel/build-embedded-jdk/builds/23 RELNOTES: None PiperOrigin-RevId: 203797576
* Adds interning to CcToolchainVariablesGravatar shahan2018-07-09
| | | | | | This entails adding equality to all VariableValue subclasses. PiperOrigin-RevId: 203796225
* Simple Markdown rendering for skydocGravatar cparsons2018-07-09
| | | | | | | This uses apache velocity engine templates to create markdown-HTML. There are other alternatives, but there is already precedent for depending on this library from docgen. RELNOTES: None. PiperOrigin-RevId: 203795431
* Allow rules to define an implicit "$toolchains" attribute which can be used toGravatar jcater2018-07-09
| | | | | | contribute Make variables. PiperOrigin-RevId: 203789736
* Bazel server, tools: ensure Readers are closedGravatar laszlocsomor2018-07-09
| | | | | | | | | | | | | | | | | | | | Follow-up to commit 59f17d6e0550bf63a0b6ef182e2d63474e058ede. Use try-with-resources to ensure Reader objects are closed eagerly. Eagerly closing Readers avoids hanging on to file handles until the garbage collector finalizes the object, meaning Bazel on Windows (and other processes) can delete or mutate these files. Hopefully this avoids intermittent file deletion errors that sometimes occur on Windows. See https://github.com/bazelbuild/bazel/issues/5512 RELNOTES: none PiperOrigin-RevId: 203771262
* Add more profiling information to better understand what's happening in ↵Gravatar twerth2018-07-09
| | | | | | | analysis phase. RELNOTES: None PiperOrigin-RevId: 203771085
* Revert "Update protobuf to 3.6.0. Fixes #5439"Gravatar Jakob Buchgraber2018-07-09
| | | | This reverts commit 6fd4e0edd4de22dec9eda13dc0b29214f2ca117e.
* Sync jmmv@'s table styles for the docs.bazel.build site.Gravatar jingwen2018-07-09
| | | | | | | | | | Sources: https://github.com/bazelbuild/bazel-blog/commit/846478d6943162f4c4d7d50001069e0ca7b2ec28 https://github.com/bazelbuild/bazel-blog/commit/aacaa25314678c08772372b3d46697f7963bb201 RELNOTES: None. PiperOrigin-RevId: 203763253
* Update protobuf to 3.6.0. Fixes #5439Gravatar Loo Rong Jie2018-07-09
|
* Revert "Update protobuf to 3.6.0. Fixes #5439"Gravatar Jakob Buchgraber2018-07-09
| | | | This reverts commit a2cac548616e6e6f433df27146c2971f352a4041.
* Update protobuf to 3.6.0. Fixes #5439Gravatar Loo Rong Jie2018-07-09
|
* Windows, CROSSTOOL: Fix feature for linking interface library and dynamic ↵Gravatar pcloudy2018-07-09
| | | | | | | | | | library 1. Do not apply /WHOLEARCHIVE to interface library 2. On Windows, we cannot link dll directly, so remove related flag_groups RELNOTES: None PiperOrigin-RevId: 203752406
* bep: introduce BuildEventArtifactUploaderFactoryGravatar buchgr2018-07-09
| | | | | | | | | | | | | | | | There can be multiple BuildEventTransports active at the same time and we need to ensure that each transport gets its own BuildEventArtifactUploader as these transports might have different lifecycles. We do that by introducing another level of indirection via the BuildEventArtifactUploaderFactory. BlazeModules now register a factory object instead of an uploader. In addition, the BuildEventArtifactUploader gets a shutdown() method that allows to free any resources associated with it. PiperOrigin-RevId: 203752092
* debian package: declare dependency on unzipGravatar Klaus Aehlig2018-07-09
| | | | | | | Fixes #5529 Change-Id: Ie3d45c4882355bd2f8b02c5afa1ad0248f28d505 PiperOrigin-RevId: 203751587
* buildkite: actually disable flaky testsGravatar buchgr2018-07-09
| | | | PiperOrigin-RevId: 203746935
* Windows,JNI: more robust nativeCreateJunctionGravatar Laszlo Csomor2018-07-09
| | | | | | | | | | | | | | | | | | | | | | | | Rewrite the CreateJunction function in the Windows JNI library. The new implementation's improvements: - succeeds if the junction already exists with the desired target; hopefully this will fix issue https://github.com/bazelbuild/bazel/issues/5433 - tolerant to concurrent filesystem modifications, e.g. if the junction's path suddenly disappears, the function reports the error correctly Fixes https://github.com/bazelbuild/bazel/issues/5433 Change-Id: I58a2314a00f6edaa7c36c35ba54616168b44eb7d Closes #5528. Change-Id: I9f5dc9237b70a433d0d8c2578a826de3d462d110 PiperOrigin-RevId: 203744515
* buildkite: disable flaky execution statistic tests on mac.Gravatar buchgr2018-07-09
| | | | | RELNOTES: None PiperOrigin-RevId: 203736658
* Improve logging for parsing skylark files.Gravatar twerth2018-07-09
| | | | | | | Also add preconditions check. RELNOTES: None. PiperOrigin-RevId: 203731929
* Delete UnionFileSystem.Gravatar lberki2018-07-09
| | | | | | | It was used in a single, Google-internal test which I cleaned up as pre-work for this change. RELNOTES: None. PiperOrigin-RevId: 203727293
* Make loading/analysis phase a lot faster.Gravatar Googler2018-07-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a fundamental bottleneck in the way targets are currently configured: The ConfiguredTargetFunction needs to load all dependent targets before it can configure its own target. Now, outgoing deps of a target can be in three states: 1. Package of dependency target not yet loaded 2. Dependency target not configured, but package already loaded 3. Dependency target configured State #3 is good, we don't need to do any more work before we can configure the current target. Now, most targets will have a combination of deps in states #1 and #2. The current behavior is to schedule the target configuration of deps in state #2 and the package loading in state #1 (to get to state #3 and #2, respectively). Only after *all* of these are computed will the corresponding ConfiguredTargetFunction be rescheduled upon which the remaining deps that were in state #1 before are now in state #2 and the corresponding target configuration can be started. This creates a substantial parallelization bottleneck by artifically sync'ing the state transitions (#1->#2 and #2->#3) for all deps. In turn, this makes the critical path artifically long. Instead, settle for just loading dependent packages as long as there are any. This is a fast single-step operation. Postpone the configuration of dependent targets to later as it can require long dependency chains to be configured. In theory, a better solution would be to split the target configuration in a way so that it is possible to ask for configured targets without yet knowing their package. However, that would require pulling config transitions apart (as some of them happen because of the origin target/config and some of them happen because of the destination target. Thus we'd need to change the ConfiguredTargetKey so that the configuration stands for the configuration that we request the target in, with target-caused transitions already applied. Now that itself leads to inefficiencies. E.g. we would start requestiong InputFiles in several configurations just to underneath always load them with a null configuration. RELNOTES: Faster analysis by improved parallelization. PiperOrigin-RevId: 203725209
* Update Blaze user manual to mention the new blaze run behavior.Gravatar diamondm2018-07-09
| | | | | RELNOTES: None. PiperOrigin-RevId: 203723885
* vfs: fix race condition in JavaIoFileSystem.deleteGravatar Laszlo Csomor2018-07-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | JavaIoFileSystem.delete() now uses Files.deleteIfExists() to delete files and empty directories. The old code used to have a race condition: a file could have been deleted or changed to a non-directory between checking it's a directory and listing its contents. WindowsFileSystem.delete() now uses the DeletePath JNI method (wrapped by WindowsFileOperations.deletePath) which is more robust than JavaIoFileSystem.delete(), because it can delete readonly files and can tolerate some concurrent filesystem modifications. Fixes https://github.com/bazelbuild/bazel/issues/5513 Change-Id: I5d2720afff8dfd3725880a6d7408d22de5935c3d Closes #5527. Change-Id: I5d2720afff8dfd3725880a6d7408d22de5935c3d PiperOrigin-RevId: 203720575
* Remove an unused method.Gravatar lberki2018-07-09
| | | | | | RELNOTES: None. PiperOrigin-RevId: 203720055
* PiperOrigin-RevId: 203687257Gravatar Googler2018-07-08
|
* Read proto javacopts directly in java_lite_proto_libraryGravatar cushon2018-07-06
| | | | | | instead of indirecting through the toolchain's compatible_javacopts. PiperOrigin-RevId: 203502116
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-29 13:26:05 +0000