| Commit message (Collapse) | Author | Age |
|
|
|
|
|
| |
but the "exec_compatible_with" attribute is present.
PiperOrigin-RevId: 201219412
|
|
|
|
|
|
| |
default.
PiperOrigin-RevId: 201218341
|
|
|
|
|
|
|
|
|
|
| |
Turns out the event handler (BlazeCommandEventHandler) prints almost all
event types, and I don't believe there's a way to tune it.
We certainly don't want these messages printed to the console unless
we're debugging the debugger, so turn them off by default.
PiperOrigin-RevId: 201211355
|
|
|
|
|
|
| |
and have DynamicCodec add to it. It's very useful to have this trail available when debugging or whitelisting.
PiperOrigin-RevId: 201205884
|
|
|
|
|
|
|
| |
The memory savings from this flag are not worth the complexity, and it interferes with action restarting.
RELNOTES: Remove support for --discard_actions_after_execution.
PiperOrigin-RevId: 201077905
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ConfiguredRuleClassProvider can specify a predicate which accepts an
OptionsDiff and the new BuildOptions in order to make a decision on whether
the given diff requires the analysis cache to be dropped.
This predicate is only called if all of the following hold:
* there was an old configuration collection (if not, the cache is not
dropped because there wasn't one yet)
* the old configuration collection is not exactly equal to the new
configuration collection (if it is, the cache is not dropped because
it definitely hasn't changed)
* the old configuration collection has the same number of configurations
as the new collection (if not, the cache is always dropped because
experimental_multi_cpu has changed, definitely not a flag which should
cause old analysis cache to stick around!)
If all of these hold, the old target configurations are paired up with
the new target configurations by index in the configuration collection,
and each pair is diffed. The predicate is called with each diff and the
corresponding new configuration's build options. If any of these
invocations returns true, the cache is dropped. Otherwise, the cache is
kept for the next build.
No implementation of this predicate is actually supplied for this
change, so the old behavior (always drop the cache if the configuration
changes at all) holds.
RELNOTES: None.
PiperOrigin-RevId: 201050049
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Runfiles.getEmptyFilenames() only considered the unconditional and
pruning manifest artifacts for empty-file insertion. Actual manifest
creation asks the empty files supplier for empty files over symlinks
(but not root symlinks!), too.
Change-Id: Ice69bbaa9e6169bff7ec5833ee7ef1b73049a4a7
Closes #5334.
Change-Id: Ice69bbaa9e6169bff7ec5833ee7ef1b73049a4a7
PiperOrigin-RevId: 201002604
|
|
|
|
|
|
| |
Moves more Action-oriented from skyframe package, which has become very unwieldy, to action package. This is needed to avoid circular dependencies caused when build-base is needed for skyframe.
PiperOrigin-RevId: 200996982
|
|
|
|
|
|
| |
multiple futures for the same fingerprint and add a test showing that racing serializations can result in duplicate writes.
PiperOrigin-RevId: 200860099
|
|
|
|
| |
PiperOrigin-RevId: 200801973
|
|
|
|
|
|
|
| |
metadata operations.
RELNOTES: None
PiperOrigin-RevId: 200765925
|
|
|
|
|
|
|
| |
metadata consumer multiple times per inline output.
RELNOTES: None
PiperOrigin-RevId: 200730252
|
|
|
|
|
| |
RELNOTES: Support for LIPO has been fully removed.
PiperOrigin-RevId: 200724578
|
|
|
|
|
|
|
|
|
| |
Ultimately, we'll need to make the call on whether these functions belong as part of the build API or as part of skylark builtins. For now, we keep them as skylark builtins.
(In either case, we'll want to migrate to @SkylarkCallable, but that's for a later change)
RELNOTES: None.
PiperOrigin-RevId: 200723605
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Background: the original code is implementing the OutputStream interface. Given a sequence of write calls, the code puts each of these writes into a queue. On the other side of the queue is an unbounded thread pool, which takes the writes off the queue one by one, and then does individual blocking writes with a fixed file offset.
There are three problems with the original code:
1. Writes are sent to the Kernel one-by-one. Imagine if the incoming writes a single-byte writes, then we do one kernel call for every single byte. This is the worst case.
2. Due to multithreading, the writes can get reordered. In the worst case, the order is reversed, and the kernel flushes each write to disk individually. Since the writes are not aligned to disk block boundaries, each write has to first *read* the block from disk, overwrite a few bytes, and then flush the block back to disk. On a spinning platter, this is the worst possible sequence of operations: write a block, read the same block back, write the same block again, read the same block back, etc., with each operation having to wait for a full disk rotation.
Note that this gets worse if there's high thread and / or disk contention, e.g., when running a build system in the background.
3. Due to the unbounded thread pool, it may end up creating a new thread for every single write (possibly as many as one per byte written). This is also the worst case, although it's probably negligible compared to 1+2.
Compared to that, this change uses an in-memory buffer before sending writes to the kernel so non-block sized writes are batched, it writes sequentially, and it uses a single thread created at the start. A single thread should be more than able to fully saturate local disk I/O, so multi-threading should ~never be a improvement here.
This might look different if we had perfectly aligned writes of an integer multiple of the disk block size to a distributed network file system or a local SSD raid. If you look at the clients of this class, that's definitely not the case: this code is primarily used for local file BEP transports - we wouldn't use local file BEP transports to write to a network file system, we'd use the BES instead. It's also used to write a couple of log files, also all local - otherwise we'd add the data to BEP. These are all unaligned, ~random-sized writes.
If we created a lot of these files, then using a thread pool with fewer threads than files and using non-blocking I/O might be an improvement due to the reduction in thread count, but I think it's very unlikely that we'll ever need that complexity.
PiperOrigin-RevId: 200694423
|
|
|
|
|
|
|
|
| |
This should be a no-op, mostly replacing PathConverter with
BuildEventArtifactUploader, since none of the implementations perform any
upload yet.
PiperOrigin-RevId: 200685325
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Note that this is not sufficient to see caching between builds on its own;
currently when any flag changes, the analysis cache is reset. A follow-up
will turn off this behavior when only test flags change while
trim_test_configuration is on.
config_settings which examine test options are treated the same as test rules;
that is, they can only be successfully analyzed at the top level or when
connected to the top level by an unbroken chain of test rules.
RELNOTES: None.
PiperOrigin-RevId: 200614584
|
|
|
|
|
|
| |
*** Reason for rollback ***
PiperOrigin-RevId: 200605975
|
|
|
|
| |
PiperOrigin-RevId: 200593618
|
|
|
|
|
|
|
| |
Deprecates the flag, though it continues to exist as a no-op so that users get a warning before errors.
RELNOTES: --noexpand_configs_in_place is deprecated.
PiperOrigin-RevId: 200572053
|
|
|
|
| |
PiperOrigin-RevId: 200561008
|
|
|
|
|
|
| |
callables that depend on CToolchain in CppConfiguration.
PiperOrigin-RevId: 200559893
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When pretty printing a Skylark value, lists are presented as the
opening bracket on a line by itself, each entry on its own line,
and the closing bracket again on its own line. While this generally
improves readability, for the empty list this is not the case, as
the expression [] can easily be understood at a glance. In fact, the
additional line even makes the outer structure harder to see, as it
is spread over even more lines. Therefore, shorten the printing of
the empty list to be on a single line.
Change-Id: I032d1550b1f99bce47dbec7e77a4d5c6656d78a1
PiperOrigin-RevId: 200558784
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
frames in thread paused events.
I started implementing conditional breakpoints server-side in
unknown commit, but:
- client-side implementations seem more common in debugging protocols
- IntelliJ in particular has great support for client-side conditional
breakpoints, but poor support for server-side
- it's unnecessary extra complexity for the server -- simple line
breakpoints + evaluation requests already give us conditional bps.
Also removing frames from thread paused events. It results in poor
performance for stepping and pausing (in which we get frame info for
potentially dozens of threads), and again isn't usual for debuggers.
Finally, ignore breakpoints when performing a client-requested
evaluation.
PiperOrigin-RevId: 200547972
|
|
|
|
|
|
|
|
|
|
|
| |
Make all external repositories depend on an additional SkyValue controllable
via commands, so support unconditional fetching of all external repositories,
as it is needed by the the `sync` command.
Improves on #5175, provides a work around for #4907.
Change-Id: I30033614c1a2fad3f1363b85ff69cf92f697c255
PiperOrigin-RevId: 200543985
|
|
|
|
|
|
|
|
|
|
| |
To disambiguate:
- @foo refers to the external dependency @foo//:foo (as before this change).
- //@foo refers to the target //@foo:@foo (i.e. in the default workspace).
RELNOTES[NEW]: Allow @ in package names.
PiperOrigin-RevId: 200541716
|
|
|
|
|
|
|
| |
Move the testing class to the tests tree. This is in preparation for
dismantling BuildView and merging the relevant parts into AnalysisPhaseRunner.
PiperOrigin-RevId: 200532794
|
|
|
|
|
|
|
|
|
| |
constraints.
Closes #5341.
Change-Id: Ib74e59fec48102469a5039e045e3f3d0e0d86d8c
PiperOrigin-RevId: 200526448
|
|
|
|
|
|
| |
We could conceivably do some monkey-patching at server startup to make all lambdas act Serializable, but one step at a time.
PiperOrigin-RevId: 200509321
|
|
|
|
|
|
| |
not serialized. Tag TestCompletionValue and any ActionExecutionValue coming from a NotifyOnActionCacheHit (i.e., tests) like that. To make such values really not shared, request the ActionExecutionValue from TestCompletionFunction as opposed to the ArtifactValue (propagating the unshareable bit up seemed like too much fuss, and I have a dream of getting rid of ArtifactValue anyway).
PiperOrigin-RevId: 200504358
|
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 200471197
|
|
|
|
|
|
|
| |
stdout/stderr.
RELNOTES: None
PiperOrigin-RevId: 200459354
|
|
|
|
|
|
|
|
| |
and output groups.
* Make ctx a keyword argument so that we can more easily add more parameters in the future and eventually remove ctx.
PiperOrigin-RevId: 200453550
|
|
|
|
|
|
| |
package lookups to get Targets.
PiperOrigin-RevId: 200452642
|
|
|
|
|
|
|
|
| |
on that future.
This allows NestedSet deserialization not to block on storage reads - in-progress deserializations are simply made a member of the NestedSets they produce.
PiperOrigin-RevId: 200440131
|
|
|
|
| |
PiperOrigin-RevId: 200410988
|
|
|
|
|
|
| |
Onto #5328
PiperOrigin-RevId: 200410170
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 200399094
|
|
|
|
|
|
|
| |
This is in preparation for dismantling BuildView and merging the relevant
parts into AnalysisPhaseRunner.
PiperOrigin-RevId: 200391088
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently the selection of C++ toolchain configuration depends on 2 different things:
1. CROSSTOOL file: a proto text file that contains multiple CrosstoolConfig.CToolchain messages, out of which we want to select the appropriate CToolchain;
2. cc_toolchain_suite rule, specified by --crosstool_top option: this rule contains "toolchains" map attribute. The map's keys are of type <cpu>|<compiler>, or just <cpu>, and the values are labels of cc_toolchain rules (which contain additional C++ information).
If there is an entry in cc_toolchain_suite.toolchains that corresponds to the --cpu and --compiler options, we use it, otherwise we loop through all the CToolchains in the CROSSTOOL file, select the one that corresponds to the --cpu and --compiler options, and then get the cc_toolchain label from cc_toolchain_suite.toolchains[toolchain.targetCpu|toolchain.compiler].
In both cases we read the CROSSTOOL file and pass all its information forward, to be used in creation of CppConfiguration and CcToolchainProvider creation.
As part of the efforts of rewriting CROSSTOOL in Skylark, we need to make obtaining the cc_toolchain label independent of the CROSSTOOL file and toolchain selection. As a step towards that goal, we add a new "toolchain_identifier" attribute to cc_toolchain, which uniquely identifies a CToolchain in the CROSSTOOL file.
Now the process of getting the CToolchain goes as follows:
Check for existence of cc_toolchain_suite.toolchains[<cpu>|<compiler>], if --compiler is specified, otherwise check for cc_toolchain_suite.toolchains[<cpu>].
1. if a value is found, load the cc_toolchain rule and look for the toolchain_identifier attribute.
1.a if the attribute exists, loop through all the CToolchains in CROSSTOOL and select the one with the matching toolchain identifier.
1.b otherwise fall back to selecting the CToolchain from CROSSTOOL by matching the --cpu and --compiler values.
2. If a value is not found, select the CToolchain from CROSSTOOL by matching the --cpu and --compiler values, and construct the key as follows: <toolchain.cpu>|<toolchain.compiler>.
In the future we will get rid of 2. by making sure that cc_toolchain_suite.toolchains[<cpu>|<compiler>] and cc_toolchain_suite.toolchains[<cpu>] are always defined.
1.b will be removed by making the cc_toolchain.toolchain_identifier attribute mandatory
After this deriving the cc_toolchain label will be independent of the CROSSTOOL file
1.a will ultimately be replaced by an attribute that points to a skylark_crosstool rule, that will contain all the information that CROSSTOOL contains, allowing us to remove CROSSTOOL altogether.
Work towards issue #5380
RELNOTES: None.
PiperOrigin-RevId: 200388550
|
|
|
|
|
|
|
|
|
|
| |
PathFragments are not well supported in Skylark, quite the opposite, the Skylark
team tries hard to not use path fragments if possible. All existing users I
looked into had to convert the PathFragment to string using str() anyway.
Because of that this cl should not break anybody.
RELNOTES: None.
PiperOrigin-RevId: 200372447
|
|
|
|
|
| |
RELNOTES:none
PiperOrigin-RevId: 200366616
|
|
|
|
| |
PiperOrigin-RevId: 200363345
|
|
|
|
|
|
|
|
|
| |
unit test coverage for this.
Note that ActionFS is not generic enough to make use of FileSystemTest.
RELNOTES: None
PiperOrigin-RevId: 200304871
|
|
|
|
|
|
|
|
| |
--experimental_android_local_test_binary_resources to true.
RELNOTES[NEW]: android_local_test now takes advantage of Robolectric's binary resource processing which allows for faster tests.
PiperOrigin-RevId: 200296572
|
|
|
|
|
|
|
|
|
| |
If this exception is thrown, performance no longer matters - we're detonating
the place and riding the explosion out to stderr. So we might as well just
dump everything we know.
RELNOTES: None.
PiperOrigin-RevId: 200290439
|
|
|
|
|
|
|
|
| |
invalidate the BUILD_INFO_KEY node on --workspace_status_command and related flag changes. Instead, the action has a supplier that allows it to retrieve the correct values at execution time.
This does not sacrifice correctness because the action executes unconditionally on every build, so it will never have stale data.
PiperOrigin-RevId: 200265375
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add a --experimental_generate_json_trace_profile option that puts a file into
the output base (or uses --profile if set).
There are still a lot of problems with this.
- unexplained holes
- way too many threads
- nonsensical event titles
- too many detail events, too little overview
- it may also cause unnecessary load
- it silently overwrites the existing file on subsequent invocations
The format is documented here: goo.gl/oMZPLh
PiperOrigin-RevId: 200259431
|
|
|
|
| |
PiperOrigin-RevId: 200247872
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
repository name is remapped.
For example if main/WORKSPACE contains:
local_repository(
name = "a",
path = "../a",
repo_mapping = {"@x" : "@y"},
)
a/BUILD
load("@x//:sample.bzl", "sample")
Then the load in a/BUILD will be resolved as "@y//:sample.bzl"
RELNOTES: None
PiperOrigin-RevId: 200227431
|