| Commit message (Collapse) | Author | Age |
|
|
|
|
| |
RELNOTES:
PiperOrigin-RevId: 204126150
|
|
|
|
| |
PiperOrigin-RevId: 204121958
|
|
|
|
|
|
|
|
|
|
| |
Calculating the throughput of a digest operation and using it to assess whether
I/O is slow or not only makes sense when we're doing sequential I/O. With
parallel hashing, it's expected that individual operations are slow due to how
the scheduler works, but it doesn't mean that the overall progress is slow.
RELNOTES: None.
PiperOrigin-RevId: 204115311
|
|
|
|
|
|
|
|
|
| |
the result of include validation if dotd file scanning (and in turn input
validation) is disabled. Fingerprinting these data structures is costly as they
are large NestedSets.
RELNOTES: None.
PiperOrigin-RevId: 204112075
|
|
|
|
|
|
|
|
|
|
|
| |
This adds support for Unix sockets to Bazel for the
remote http cache. See corresponding issue #5098
for discussion.
RELNOTES: Introduce the --remote_cache_proxy flag,
which allows for remote http caching to connect
via a unix domain socket.
PiperOrigin-RevId: 204111667
|
|
|
|
|
|
|
| |
seem to be used.
RELNOTES: None.
PiperOrigin-RevId: 204084726
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Breaks non-Rabbit builds, see b/111275650.
*** Original change description ***
Avoid long, duplicated directory structures. In the common case, generated
files are going to be beneath the target that generates them. In this case,
don't duplicated the package's path.
RELNOTES: None.
PiperOrigin-RevId: 204084475
|
|
|
|
|
|
|
|
| |
This observably removes any ill effect of CAS transience.
Closes #5229.
PiperOrigin-RevId: 204010317
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 203977219
|
|
|
|
|
|
|
| |
This will be needed by py_wrap_cc in a follow up CL.
RELNOTES:none
PiperOrigin-RevId: 203964457
|
|
|
|
|
| |
Change-Id: Iefeab0de85541b963c6cc262339e73b79f630f8b
PiperOrigin-RevId: 203952823
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Refocus synchronization mechanism to cope with file descriptor set fork-
induced races to more tightly constrain concurrent fork/exec pairs. This
problem has been observed in bazel proper repeatedly, exhibiting as the
iconic ETXTBSY - Text file busy in wide worker pool builds and tests.
Evidence that this was discovered by @buchgr is in the comment and change
to the embedded ExecutionService implementation, and the description of
the race and the need for the synchronization was lifted from that scope
to the JavaSubprocessFactory. This factory is a singleton and represents
the gateway to all worker process execution, and serves as the correct
lock primitive to ensure that file descriptor sets are not duplicated
across forks, which gave rise to this issue.
To test this, I demonstrated a reproducer presented at
https://bugs.java.com/view_bug.do?bug_id=8068370
with 2.4% of invocations in that pathological case exhibiting the issue.
With a functionally equivalent change - synchronizing around a
processBuilder.start() call - as the only modification to the reproducer,
no further failures of any kind were observed, over several hundred runs.
Closes #5556.
PiperOrigin-RevId: 203947224
|
|
|
|
|
|
| |
RELNOTES:
--experimental_shortened_obj_file_path is removed.
PiperOrigin-RevId: 203944948
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
compose a nested set encompassing both). The current prunableHeaders were used
in various places, at least two of which let to a duplicate iteration through
the comparatively large declaredIncludeSrcs: CppCompileAction.computeKey() and
CppCompileAction.getAllowedDerivedInputs().
Also do some other minor optimizations:
- CcCompileActionBuilder.buildAllInputs() now just returns a NestedSet with the
same order as its parameter. As inputsForValidation is almost always empty,
this makes this NestedSet pass the underlying one through.
- CcCompilationContext.directModuleMaps is just a list of the module maps of
direct dependencies. As such, there is no use for keeping them in a
NestedSet.
RELNOTES: None.
PiperOrigin-RevId: 203938011
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
commit 59f17d6e0550bf63a0b6ef182e2d63474e058ede updated files to use
try-with-resources when dealing with streams.
The change also got rid of asByteSource, replacing
it with throw-away ByteSource instances wrapping a
try-with-resources-guarded InputStream. Doing so
was unnecessary though, because all ByteSource
methods (except for open[Buffered]InputStream)
close the stream. Thanks to @jbduncan to point
that out and explain in detail [1].
[1] see comment thread on https://github.com/bazelbuild/bazel/commit/59f17d6e0550bf63a0b6ef182e2d63474e058ede under `FilesetManifest.java`
RELNOTES: none
PiperOrigin-RevId: 203934830
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 203934582
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Follow-up to commit 09d20311d982606093ed881d779bb05a5ee70ed3.
Use try-with-resources to ensure Writer objects
are closed eagerly.
Eagerly closing Writers avoids hanging on to
file handles until the garbage collector finalizes
the object, meaning Bazel on Windows (and
other processes) can delete or mutate these files.
Hopefully this avoids intermittent file deletion
errors that sometimes occur on Windows.
See https://github.com/bazelbuild/bazel/issues/5512
RELNOTES: none
PiperOrigin-RevId: 203934471
|
|
|
|
|
|
|
| |
Also introduce NamedThreadSafeOutputFormatterCallback for common code.
RELNOTES: None
PiperOrigin-RevId: 203932877
|
|
|
|
|
|
| |
Fixes #5439.
PiperOrigin-RevId: 203917717
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Don't use NestedSets of NestedSets. Not sure whether this is a performance
improvement or has serialization benefits, but it does make understanding the
structure a lot easier.
- Use the same large NestedSet for all header information (whether or not they
are modular and what pregrepped headers might exist). This speeds up the
include scanner calls getLegalGeneratedScannerFileMap and getModularHeaders
because they now iterate over just a single NestedSet and the second
evaluation of a NestedSet is much cheaper. In a subsequent change, we can
likely fold all three iterations over this NestedSet into one (in
getLegalGeneratedScannerFileMap, getModularHeaders and getUsedModules).
Measurements show a small reduction in heap usage.
RELNOTES: None.
PiperOrigin-RevId: 203909308
|
|
|
|
|
|
|
| |
ToolchainContext configuration earlier.
Change-Id: I034f56a8a181aac19ecb38e51558ccdc6c0c7a6a
PiperOrigin-RevId: 203835978
|
|
|
|
|
|
|
| |
error messaging and commenting when there's an unresolvable skylark type.
RELNOTES: None.
PiperOrigin-RevId: 203826504
|
|
|
|
| |
PiperOrigin-RevId: 203813666
|
|
|
|
| |
PiperOrigin-RevId: 203813658
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
dependencies.
This is a reland of https://github.com/bazelbuild/bazel/commit/5deca4cf88f5568771f2c836a9b8c693b88bd749.
This will make protoc see as direct dependencies the .proto files that were included using the proto_source_root flag.
Until now, Bazel passed to protoc the direct dependencies of a target as the path relative to the WORKSPACE, which made it fail when a shorter path, relative to the package was used.
Progress on #4544.
RELNOTES: None.
PiperOrigin-RevId: 203808292
|
|
|
|
|
|
| |
This entails adding equality to all VariableValue subclasses.
PiperOrigin-RevId: 203796225
|
|
|
|
|
|
| |
contribute Make variables.
PiperOrigin-RevId: 203789736
|
|
|
|
|
|
|
| |
analysis phase.
RELNOTES: None
PiperOrigin-RevId: 203771085
|
|
|
|
|
|
|
|
|
|
| |
Sources:
https://github.com/bazelbuild/bazel-blog/commit/846478d6943162f4c4d7d50001069e0ca7b2ec28
https://github.com/bazelbuild/bazel-blog/commit/aacaa25314678c08772372b3d46697f7963bb201
RELNOTES: None.
PiperOrigin-RevId: 203763253
|
| |
|
|
|
|
| |
This reverts commit a2cac548616e6e6f433df27146c2971f352a4041.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
library
1. Do not apply /WHOLEARCHIVE to interface library
2. On Windows, we cannot link dll directly, so remove related flag_groups
RELNOTES: None
PiperOrigin-RevId: 203752406
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There can be multiple BuildEventTransports active at
the same time and we need to ensure that each transport
gets its own BuildEventArtifactUploader as these transports
might have different lifecycles.
We do that by introducing another level of indirection via
the BuildEventArtifactUploaderFactory. BlazeModules now
register a factory object instead of an uploader.
In addition, the BuildEventArtifactUploader gets a shutdown()
method that allows to free any resources associated with it.
PiperOrigin-RevId: 203752092
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rewrite the CreateJunction function in the Windows
JNI library.
The new implementation's improvements:
- succeeds if the junction already exists with the
desired target; hopefully this will fix issue
https://github.com/bazelbuild/bazel/issues/5433
- tolerant to concurrent filesystem modifications,
e.g. if the junction's path suddenly disappears,
the function reports the error correctly
Fixes https://github.com/bazelbuild/bazel/issues/5433
Change-Id: I58a2314a00f6edaa7c36c35ba54616168b44eb7d
Closes #5528.
Change-Id: I9f5dc9237b70a433d0d8c2578a826de3d462d110
PiperOrigin-RevId: 203744515
|
|
|
|
|
|
|
| |
Also add preconditions check.
RELNOTES: None.
PiperOrigin-RevId: 203731929
|
|
|
|
|
|
|
| |
It was used in a single, Google-internal test which I cleaned up as pre-work for this change.
RELNOTES: None.
PiperOrigin-RevId: 203727293
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There is a fundamental bottleneck in the way targets are currently configured:
The ConfiguredTargetFunction needs to load all dependent targets before it can
configure its own target. Now, outgoing deps of a target can be in three
states:
1. Package of dependency target not yet loaded
2. Dependency target not configured, but package already loaded
3. Dependency target configured
State #3 is good, we don't need to do any more work before we can configure the
current target. Now, most targets will have a combination of deps in states #1
and #2. The current behavior is to schedule the target configuration of deps in
state #2 and the package loading in state #1 (to get to state #3 and #2,
respectively). Only after *all* of these are computed will the corresponding
ConfiguredTargetFunction be rescheduled upon which the remaining deps that were
in state #1 before are now in state #2 and the corresponding target
configuration can be started.
This creates a substantial parallelization bottleneck by artifically sync'ing
the state transitions (#1->#2 and #2->#3) for all deps. In turn, this makes the
critical path artifically long. Instead, settle for just loading dependent
packages as long as there are any. This is a fast single-step operation.
Postpone the configuration of dependent targets to later as it can require long
dependency chains to be configured.
In theory, a better solution would be to split the target configuration in a
way so that it is possible to ask for configured targets without yet knowing
their package. However, that would require pulling config transitions apart (as
some of them happen because of the origin target/config and some of them happen
because of the destination target. Thus we'd need to change the
ConfiguredTargetKey so that the configuration stands for the configuration that
we request the target in, with target-caused transitions already applied. Now
that itself leads to inefficiencies. E.g. we would start requestiong InputFiles
in several configurations just to underneath always load them with a null
configuration.
RELNOTES: Faster analysis by improved parallelization.
PiperOrigin-RevId: 203725209
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
JavaIoFileSystem.delete() now uses
Files.deleteIfExists() to delete files and empty
directories.
The old code used to have a race condition: a
file could have been deleted or changed to a
non-directory between checking it's a directory
and listing its contents.
WindowsFileSystem.delete() now uses the
DeletePath JNI method (wrapped by
WindowsFileOperations.deletePath) which is more
robust than JavaIoFileSystem.delete(), because it
can delete readonly files and can tolerate some
concurrent filesystem modifications.
Fixes https://github.com/bazelbuild/bazel/issues/5513
Change-Id: I5d2720afff8dfd3725880a6d7408d22de5935c3d
Closes #5527.
Change-Id: I5d2720afff8dfd3725880a6d7408d22de5935c3d
PiperOrigin-RevId: 203720575
|
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 203720055
|
| |
|
|
|
|
|
|
|
| |
This metric measures the used heap size at the end of the build, after a GC. Because this is expensive it is only collected if a flag is passed.
RELNOTES:
PiperOrigin-RevId: 203492365
|
|
|
|
|
| |
RELNOTES: [JavaInfo] Outputs are merged in java_common.merge().
PiperOrigin-RevId: 203474741
|
|
|
|
|
|
|
| |
It's no longer needed.
RELNOTES: None.
PiperOrigin-RevId: 203469119
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 203466429
|
|
|
|
|
|
|
|
|
|
| |
In this way, their name is one they can actually be referred to by.
This is necessary, as Skylark repository rules end up in the resolved
file when freezing the versions of external dependencies---and the call
to the repository function might be indirect.
Change-Id: Ie1b5a28fac4f690c424de848b42d0c4a217d494d
PiperOrigin-RevId: 203466286
|
|
|
|
|
|
|
| |
want to use input discovery when building modules for performance reasons.
RELNOTES: None.
PiperOrigin-RevId: 203463009
|
|
|
|
|
|
|
|
|
|
|
| |
building with C++ modules, .d file scanning is not effective as Clang (only
compiler supporting Bazel's version of C++ modules) reads all of the files
passed in anyway.
This is not meant to be activated yet, but for running experiments.
RELNOTES: None.
PiperOrigin-RevId: 203459168
|
|
|
|
|
|
|
|
|
|
|
| |
Commit d3f7f7ae introduced a bug in the FileTransport implementations
where events might not be written to file in the correct order. This
change fixes this.
Fixes #5515
RELNOTES: None
PiperOrigin-RevId: 203457636
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 203454741
|