| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
| |
This cl adds following flags:
* --experimental_disable_legacy_cc_compilation_api
* --experimental_disable_legacy_cc_linking_api
* --experimental_disable_linking_mode_flags
* --experimental_disable_compilation_mode_flags
* --experimental_disable_legacy_crosstool_fields
RELNOTES: None
PiperOrigin-RevId: 204924599
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
range used to use MutableList which would eagerly allocate an array list with
all range elements, which is not efficient for very large ranges or when only
a small number of its elements are used.
This implementation uses a constant amount of RAM and computes a value for
each requested index.
For the following Skylark snippet:
```
def check_content(t):
if t == []:
return t
return False
def modulo(n):
return n % 797
N = 10000000
[check_content(i) for i in range(N)]
[check_content(i) for i in range(N)]
[modulo(i) for i in range(N)]
[modulo(i) for i in range(N)]
```
the total runtime goes from
```
$ time bazel-bin/src/main/java/com/google/devtools/skylark/Skylark test.bzl
bazel-bin/src/main/java/com/google/devtools/skylark/Skylark test.bzl 93.09s user 1.67s system 316% cpu 29.930 total
```
to
```
$ time bazel-bin/src/main/java/com/google/devtools/skylark/Skylark test.bzl
bazel-bin/src/main/java/com/google/devtools/skylark/Skylark test.bzl 31.45s user 0.86s system 179% cpu 17.974 total
```
which reflects the reduced system time (fewer allocations) and performance.
Closes #5240.
PiperOrigin-RevId: 204918577
|
|
|
|
|
|
|
|
|
|
|
| |
The synchronized block in sendBuildEvent is
not needed at all, since all the logic in
there is local to the method and thread safe.
See the issue for more details.
RELNOTES: None
PiperOrigin-RevId: 204915541
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a local file upload fails, then the build event
gets lost. This was the case because of a bug in the
implementation where local file upload errors are
retried by the BES retrier. This is the root cause
for the "Non-consecutive sequence number" errors we
have been seeing recently.
RELNOTES: None
PiperOrigin-RevId: 204906550
|
|
|
|
|
|
|
| |
to the include scanner and slightly reshuffle code.
RELNOTES: None.
PiperOrigin-RevId: 204906167
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The Bazel client on Windows now writes extracted
binaries to disk in parallel. On all other systems
it writes them serially (as before).
This change makes blaze.cc:ActuallyExtractData()
about 3x faster when using a HDD. (In previous
experiments I saw no speedup with multi-threaded
writing on machines with an SSD.)
The Windows-specific code uses the native
Threadpool API of Windows, creating a pool of at
least 8 and at most 16 threads. (This seems to be
a good balance between speed and thread count.)
The OS manages everything about the pool; Bazel
submits callbacks and the pool executes them
asynchronously.
blaze.cc:ActuallyExtractData() speed, before:
- Windows: 6.48s (avg) / 6.38s (median)
- Linux (Debian): 4.78s (avg) / 4.79s (median)
blaze.cc:ActuallyExtractData() speed, after:
- Windows (8-16 threads): 2.05s (avg) / 2.01s (md)
- Windows (1 thread): 5.77s (avg) / 5.74s (median)
See https://github.com/bazelbuild/bazel/issues/5444
Change-Id: I7211f3d28eb8b9837352c16ff8df0411d5a9ebe1
Closes #5600.
Change-Id: I7a74d62a563c92948a4dfa8ad5ac83eae018db10
PiperOrigin-RevId: 204891217
|
|
|
|
| |
PiperOrigin-RevId: 204889082
|
|
|
|
| |
PiperOrigin-RevId: 204882046
|
|
|
|
|
|
|
|
|
|
|
| |
Treat repository rules the same way as build rules: they may only be used,
if exported by a Skylark file. It has never been intended to create external
repositories by anonymous rules, and, in fact, for properly recording resolved
information, it is necessary that all repository rules used can be referred
to by an accessible name.
Change-Id: Ib9259d58be66b033721a6f591656c45889f49931
PiperOrigin-RevId: 204872735
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Depot has been fixed.
RELNOTES[INC]: If the same artifact is generated by two distinct but identical actions, and a downstream action has both those actions' outputs in its inputs, the artifact will now appear twice in the downstream action's inputs. If this causes problems in Skylark actions, you can use the uniquify=True argument in Args.add_args.
PiperOrigin-RevId: 204827477
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 204817013
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 204809750
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 204801047
|
|
|
|
|
|
|
| |
parameters of the skylark attr module.
RELNOTES: None.
PiperOrigin-RevId: 204797954
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Tulsi uses BEP json output in its UI, to simulate Bazel terminal outptut. This
means we have to promptly flush the stream. It's sufficient to do flushing at
the granularity of whole events, not any specific count of bytes, since whole
events are what's being consumed by the user.
To balance IO throughput and interactivity, let's flush at a regular
sub-second interval.
(The alternative solution of using a stream with smaller buffer could still
end up with small-sized event descriptions buffered arbitrarily long; and
abandoning buffering altogether would be suboptimal for throughput when
writing a long sequence of small build events.)
RELNOTES: None.
PiperOrigin-RevId: 204790794
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 204788478
|
|
|
|
|
|
|
| |
Temporary solution finding the SkylarkCallable fields using reflection,
mirroring the existing handling of ClassObject.
PiperOrigin-RevId: 204779295
|
|
|
|
|
|
|
|
|
|
| |
referencing the main repository using its name caused bazel to treat it as a separate external repository.
Closes #5586.
Fixes #3115.
RELNOTES: None
PiperOrigin-RevId: 204752150
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
non-hermetic events that happen as part of repository rules).
Defining representation for Execute events for workspace logging.
In the future:
- Add more events
- Allowing to specify log file rather than dumping to INFO
- Log levels, full or alerts only
RELNOTES: None
PiperOrigin-RevId: 204748436
|
|
|
|
|
|
|
|
|
|
|
| |
This was providing runtime libraries. This provider is redundant because the
same libraries are in CcLinkParamsStore.
Note to sheriff: Shouldn't break anything. If any, breakages would most likely be in Go test rules. I will look at the nightly tomorrow anyway. The breakages in the linked TGP re-run are not caused by this change, later re-runs have unrelated changes.
TESTED=[]
RELNOTES:none
PiperOrigin-RevId: 204738136
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- CcLinkingInfo has a getter for CcLinkParams.
- Artifacts and linkopts of CcLinkParams are accessible.
- CcCompilationInfo constructor now accepts defines and include dirs.
- CcCompilation now has getters for headers, defines and include dirs.
CcCompiilationInfo(
headers=depset([Artifacts]),
defines=depset([Strings]),
include_dirs=depet([Strings])
CcCompilationInfo.headers
CcCompilationInfo.defines
CcCompilationInfo.include_dirs
--
cc_common.merge([CcLinkingInfos]) returns CcLinkingInfo
--
CcLinkingInfo.static_shared_params
CcLinkingInfo.static_no_shared_params
CcLinkingInfo.no_static_shared_params
CcLinkingInfo.no_static_no_shared_params
--
CcLinkParams.libraries
CcLinkParams.linkopts
CcLinkParams.dynamic_libraries_for_runtime
RELNOTES:none
PiperOrigin-RevId: 204700779
|
|
|
|
|
|
|
| |
are difficult or impossible to verify by annotation processor.
RELNOTES: None.
PiperOrigin-RevId: 204540521
|
|
|
|
|
|
|
|
|
| |
files which define new provider types and instantiate Labels.
Also improve docstring usage in attr definitions by doing whitespace trimming.
RELNOTES: None.
PiperOrigin-RevId: 204537623
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 204533485
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 204531667
|
|
|
|
|
|
|
| |
https://github.com/bazelbuild/bazel/commit/b39c69394c5e7991ca8d04efac2142c22947a7c5
RELNOTES: None
PiperOrigin-RevId: 204522278
|
|
|
|
|
| |
RELNOTES: none
PiperOrigin-RevId: 204514591
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 204514384
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 204505003
|
|
|
|
|
|
|
| |
This old provider will be deprecated, and removing documentation for this type is of higher priority as it conflicts with the documentation for "actions", another skylark type.
RELNOTES: None.
PiperOrigin-RevId: 204480354
|
|
|
|
|
|
|
| |
Previously, any absolute label (starting with double slash) would be converted to an absolute path, instead of a path relative to the current workspace directory.
RELNOTES: None.
PiperOrigin-RevId: 204472080
|
|
|
|
|
|
|
| |
Java code.
RELNOTES: None
PiperOrigin-RevId: 204471346
|
|
|
|
|
|
|
| |
test hooks.
RELNOTES: None
PiperOrigin-RevId: 204468647
|
|
|
|
|
|
| |
RELNOTES:
No longer define G3_VERSION_INFO for c++ linkstamp compiles, as it was a duplicate of G3_TARGET_NAME.
PiperOrigin-RevId: 204466459
|
|
|
|
|
| |
RELNOTES:none
PiperOrigin-RevId: 204463998
|
|
|
|
|
|
|
| |
It was not opensourced, and even internally was not used. And we hate having internal-only code.
RELNOTES: None.
PiperOrigin-RevId: 204441702
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
CreateJunction and DeletePath are now more
resilient to errors:
- CreateJunction opens the junction path to check
its target requesting fewer rights and with
greater sharing permission. This way it can
check junction targets even if the junction name
is opened by another process with no sharing.
- DeletePath attempts to call FindFirstFileW if
GetFileAttributesW fails with
ERROR_ACCESS_DENIED. There's hardly any info
about this error mode online, except for a code
comment in the .NET CoreFX library. (See new
code comments in this commit.)
Also:
- Change the error codes for DeletePath.
- Wrap the DeletPath error codes in a struct for
better readability.
Fixes https://github.com/bazelbuild/bazel/issues/5433
Change-Id: I5b6e0f27b5b22c1cf00da90104495eda84178283
Closes #5590.
Change-Id: I5b6e0f27b5b22c1cf00da90104495eda84178283
PiperOrigin-RevId: 204438994
|
|
|
|
|
|
|
|
| |
available.
The owning labels are the labels of the top-level configured targets that requested this artifact to be built (there may be many such targets). In cases where the artifact is added not through a configured target (build-info artifacts and coverage artifacts), the label of the artifact's owner is used.
PiperOrigin-RevId: 204432951
|
|
|
|
|
|
|
|
|
| |
ResolvedFile objects.
While this does not eliminate the need for stat operation yet, it gets rid of one usage of the stat result (and is also a self contained change)
RELNOTES: None
PiperOrigin-RevId: 204417559
|
|
|
|
|
|
|
| |
preceding integer when serializing BuildOptions.DiffForReconstruction.
Reorder the cache map inserts due to a subtle race condition that can occur.
PiperOrigin-RevId: 204376273
|
|
|
|
|
|
|
| |
toolchain API
RELNOTES: None.
PiperOrigin-RevId: 204291210
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
contain a large set of files required for a particular toolchain that are added
to every single compile action. Some exeutors in turn sort all action inputs in
order to properly cache, deduplicate and serialize them.
Sorting the expansion of these middlemen needs to be done only once and a later
sorting together with a bunch of other action input files is much faster (as
Java's TimSort algorithm is effectively linear with mostly sorted inputs).
RELNOTES: None.
PiperOrigin-RevId: 204285426
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change allows local files referenced by the BEP/BES protocol
to be uploaded to a ByteStream gRPC service.
The ByteStreamUploader is now implicitly also used by the BES
module which has a different lifecycle than the remote module.
We introduce reference counting to ensure that the channel is
closed after its no longer needed. This also fixes a bug where
we currently leak one socket per remote build until the Bazel
server is shut down.
RELNOTES: None
PiperOrigin-RevId: 204275316
|
|
|
|
|
|
|
|
|
|
|
| |
- Don't duplicate usedModules into additionalInputs (this shouldn't be
necessary).
- Use ImmutableLists instead of ImmutableSets where possible to reduce memory
consumption.
- Use set operations to make the code more readable.
RELNOTES: None.
PiperOrigin-RevId: 204268489
|
|
|
|
|
|
|
|
| |
They're different anyway and we will need this to set a different default value
for aquery.
RELNOTES: None
PiperOrigin-RevId: 204259933
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Breaks //devtools/blaze/integration:{[]_test_test,gdp_validation_test} and at leats //contentads/supermixer/server:supermixer .
*** Original change description ***
Refactor handling of API generation in JavaPluginInfoProvider
Instead of keeping two copies of state for the API-generating and
non-API-generating cases, create a 'JavaPluginInfo' abstraction to contain all
state for each case, and then keep two copies in the top-level
JavaPluginInfoProvider provider.
This will make it easier and less error-prone to add additional state to the
provider.
PiperOrigin-RevId: 204258844
|
|
|
|
| |
PiperOrigin-RevId: 204254234
|
|
|
|
|
|
|
|
| |
delay the string conversion till we actually write the manifest file.
This might get some memory savings after adding some in unknown commit.
RELNOTES: None
PiperOrigin-RevId: 204216582
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
than the graph version when that is feasible.
* It's not feasible when the computation accesses outside state, i.e. is non-hermetic, so see below.
* It's also more complicated (and not worth the trouble) when the computation is taking place just for the error status.
Have SkyFunctionName declare whether the function it corresponds to is hermetic or non-hermetic. Only non-hermetically-generated SkyValues can be directly marked changed, and non-hermetic SkyFunctions have their values saved at the graph version, not the max of the child versions. All SkyFunctions are hermetic except for the ones that can be explicitly dirtied.
A marked-hermetic SkyFunction that has a transient error due to filesystem access can be re-evaluated and get the correct version: if it throws an IOException at version 1 and then, when re-evaluated at version 2 with unchanged dependencies, has a value, the version will be version 1.
All Skyframe unit tests that were doing non-hermetic things to nodes need to declare that those nodes are non-hermetic. I tried to make the minimal set of changes there, so that we had good incidental coverage of hermetic+non-hermetic nodes. Also did some drive-by clean-ups around that code.
Artifacts are a weird case, since they're doing untracked filesystem access (for source directories). Using max(child versions) for them gives rise to the following correctness bug: 1. do a build at v1 that creates a FileStateValue for dir/ at v1. Then at v2, add a file to dir/ and do a build that consumes dir/ as a source artifact. Now the artifact for dir/ will (incorrectly) have v1. Then at v1, do that build again. We'll consume the "artifact from the future". However, this can only have an effect when using the local action cache, since the incorrect value of the artifact (the mtime) is only consumed by the action cache. Bazel is already broken in this way (incremental builds don't invalidate directories), so this change doesn't make things worse.
PiperOrigin-RevId: 204210719
|