| Commit message (Collapse) | Author | Age |
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 208861642
|
|
|
|
|
|
|
|
|
|
| |
Skyframe.
This avoids some unnecessary iteration over already-emitted events that can show up in profiles, and allows us to store execution-phase values a bit more compactly, since we don't need to carry around wrapper objects and nested sets everywhere.
This crucially depends on the fact that we can't build a target in the execution phase without first having analyzed it in a separate Skyframe call. Skyframe normally propagates all events/posts up the graph because it must be able to emit them if a user requests a node that only transitively depends on the node that emitted an event. However, because we do analysis in a separate Skyframe call, any warnings/posts associated with the analysis nodes will be emitted then, and we don't need to propagate them into execution.
PiperOrigin-RevId: 208767078
|
|
|
|
|
|
| |
TransitiveVersionTable uses futures.
PiperOrigin-RevId: 208402465
|
|
|
|
|
|
|
|
|
|
| |
conflicts if the current set of configured targets is not a subset of the largest set of configured targets that have been checked for conflicts.
Also rework the flow between SkyframeBuildView and SkyframeActionExecutor to remove the SkyframeExecutor middleman.
Also reword the error message in case of an ArtifactPrefixConflictException, since a clean should no longer be necessary.
PiperOrigin-RevId: 207322139
|
|
|
|
|
|
|
| |
looked-up artifact values.
RELNOTES: None
PiperOrigin-RevId: 207295716
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a new PrepareAnalysisPhaseFunction, which started out as a copy of
some existing code from SkyframeExecutor, BuildView, AnalysisPhaseRunner,
AnalysisUtils, and ConfigurationResolver, which was then modified to work
inside Skyframe.
Most of our tests already work with the new code, except for some of the tests
related to configuration trimming in combination with dependency cycles. The
reason for this is that we can only recover from dependency cycles at the end
of a Skyframe invocation, but never inside a Skyframe invocation. The new code
therefore cannot return partial results like the old code.
This seems to make null builds a bit faster. In my testing, I saw null build
times for a single test target go from ~50ms to ~40ms. This is probably due to
slightly better caching - it seems that computing the configuration transitions
and top-level targets is non-negligible, even if there's only a single
top-level configuration for a single top-level target.
This might be an even bigger win if there are a lot of top-level targets and
configurations.
PiperOrigin-RevId: 207083192
|
|
|
|
|
|
|
| |
Previously, two successive builds of targets A and B would fail if the two sets had conflicting actions. Now, only fail such builds if the targets from the current build has action conflicts.
RELNOTES: None
PiperOrigin-RevId: 206974011
|
|
|
|
|
|
|
|
| |
out of it.
Setting up the package cache is necessary for proper loading. In the future, BuildConfiguration.Option options will be targets that need to be loaded in order to be parsed. Thus, we will need to be able to do loading before BuildConfiguration.Options are parsed.
PiperOrigin-RevId: 206964473
|
|
|
|
| |
PiperOrigin-RevId: 206568465
|
|
|
|
|
|
|
|
| |
Instead of using the default thread pool size of 200, use the number set for
the loading phase. This is in preparation for interleaving the loading and
target pattern eval phases.
PiperOrigin-RevId: 206172915
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add an option to provide a file with a resolved value, that will be
used to verify that the repositories mentioned in this file produce
a correct directory tree.
RELNOTES: newly added options --experimental_repository_hash_file and
--experimental_verify_repository_rules allow to verify for repositories
the directory generated against pre-recorded hashes. See documentation
for those options.
Work towards #5660.
Change-Id: I2d8becb188d0fa51e890fb8f6139f321cca14b7b
PiperOrigin-RevId: 206016792
|
|
|
|
|
|
|
|
|
|
|
| |
Instead, refactor the code to use TargetPatternPhaseValue exclusively. This
removes the need to convert from TargetPatternPhaseValue to LoadingResult, and
prepares for interleaving.
It also reduces the number of Skyframe calls which may speed up null builds a
bit, as a followup for https://github.com/bazelbuild/bazel/commit/1067310e18cb9ac203110726de0be53bdc403cea.
PiperOrigin-RevId: 205989338
|
|
|
|
| |
PiperOrigin-RevId: 205876673
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Fixed duplicate derived inputs bug. Test is in diffbase.
RELNOTES[INC]: If the same artifact is generated by two distinct but identical actions, and a downstream action has both those actions' outputs in its inputs, the artifact will now appear twice in the downstream action's inputs. If this causes problems in Skylark actions, you can use the uniquify=True argument in Args.add_args.
PiperOrigin-RevId: 205863806
|
|
|
|
|
|
|
|
|
|
|
| |
Also simplify LoadingPhaseCompleteEvent, and SkyframeExecutor, and remove
LoadingCallback, which is unnecessary now that we only have a single
implementation (previously LoadingPhaseRunner).
This also removes some of the excessive Skyframe calls introduced by
https://github.com/bazelbuild/bazel/commit/1067310e18cb9ac203110726de0be53bdc403cea, and prepares for interleaving target pattern eval and loading.
PiperOrigin-RevId: 205813197
|
|
|
|
|
|
|
|
| |
Default value is true, and behavior related to //tools/defaults package is not
changed. If set it to false, then in-memory Dfaultpacked will not be created.
RELNOTES:none
PiperOrigin-RevId: 205643628
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Still bugs lurking. See linked bug.
*** Original change description ***
Automated rollback of commit eb587075b0d6ffab1cf9e69ede1b7e547905e547.
*** Reason for rollback ***
Depot has been fixed.
RELNOTES[INC]: If the same artifact is generated by two distinct but identical actions, and a downstream action has both those actions' outputs in its inputs, the artifact will now appear twice in the downstream action's inputs. If this causes problems in Skylark actions, you can use the uniquify=True argument in Args.add_args.
PiperOrigin-RevId: 204997569
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Depot has been fixed.
RELNOTES[INC]: If the same artifact is generated by two distinct but identical actions, and a downstream action has both those actions' outputs in its inputs, the artifact will now appear twice in the downstream action's inputs. If this causes problems in Skylark actions, you can use the uniquify=True argument in Args.add_args.
PiperOrigin-RevId: 204827477
|
|
|
|
|
| |
Change-Id: Iefeab0de85541b963c6cc262339e73b79f630f8b
PiperOrigin-RevId: 203952823
|
|
|
|
|
|
| |
nodes, instead of invalidating them. Invalidating isn't sound, because we're depending on externally injected state to be re-read during evaluation. Also rename the method to resetConfigurationCollectionForTesting since it's only used in tests.
PiperOrigin-RevId: 203172895
|
|
|
|
|
|
| |
ArtifactResolverSupplier in SkyframeExecutor.
PiperOrigin-RevId: 201705857
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Breaks Javascript compilation.
There's currently no way to iterate over a depset of Artifacts and deduplicate by identical paths in Skylark. This means that actions that want to do something once per Artifact in a depset (add a flag to the command line with the path of the Artifact for instance) will have duplicate entries if there are multiple Artifacts with the same path.
This is not a true automated rollback, since I tried to make this as minimal as possible, to avoid merge conflicts and keep some of the benefits of the rolled back CL. In particular, the tests that were changed in the original CL to give artifacts their correct owners did not need to be changed back, since the owners are still correct.
Moreover, this effectively contains rollbacks of unknown commit and https://github.com/bazelbuild/bazel/commit/39d6f89644107a8f7c080c4f4aec8527c1a73954, but keeps test coverage from those CLs as well. Comments added to CL thread where not a clean rollback (there are plenty of files not changed at all: ActionArtifactCycleReporter is the main wart, since it can still handle SkyKeys with Artifact as the argument, but Artifact is no longer the argument to any SkyKey).
RELNOTES: None
*** Original change description ***
Make Artifact#equals take the owner into account for derived artifacts.
Derived artifacts' owners are important because they are used to determine the artifact's generating action. Source artifacts' owners are not used in this way, so I left them alone.
This allows us to get rid of most uses of ArtifactSkyKey. We may be able to delete it entirely in a follow-up.
PiperOrigin-RevId: 201464780
|
|
|
|
|
|
|
| |
To start we add just a single metric, the number of actions constructed in the current build.
RELNOTES: None
PiperOrigin-RevId: 201248490
|
|
|
|
|
|
|
| |
The memory savings from this flag are not worth the complexity, and it interferes with action restarting.
RELNOTES: Remove support for --discard_actions_after_execution.
PiperOrigin-RevId: 201077905
|
|
|
|
|
|
| |
Moves more Action-oriented from skyframe package, which has become very unwieldy, to action package. This is needed to avoid circular dependencies caused when build-base is needed for skyframe.
PiperOrigin-RevId: 200996982
|
|
|
|
|
| |
RELNOTES: Support for LIPO has been fully removed.
PiperOrigin-RevId: 200724578
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The intent is that the main thread has ~zero gaps in the profile (though there
may still be small gaps due to the time between one try block and an
immediately subsequent try block).
We need to be careful not to wrap markPhase calls (or methods that call
markPhase) in try blocks for the profiler - the Profiler requires that all
markPhase calls happen at top level, and throws an exception if not.
This should not have any performance impact - all of these are once per build,
or at most once per module per build, and we don't expect a very large number
of modules (and if we see an increasing number, we need to change the module
API to not have to call every single module, but only those that are actually
interested in certain events, maybe with an EventBus-based setup).
PiperOrigin-RevId: 200712677
|
|
|
|
|
|
|
|
| |
invalidate the BUILD_INFO_KEY node on --workspace_status_command and related flag changes. Instead, the action has a supplier that allows it to retrieve the correct values at execution time.
This does not sacrifice correctness because the action executes unconditionally on every build, so it will never have stale data.
PiperOrigin-RevId: 200265375
|
|
|
|
| |
PiperOrigin-RevId: 200210007
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- the TargetPatternPreloader is still used for query in all its forms
- the remaining TargetPatternEvaluator part is no longer used except in tests
- also make both implementations stateless and pass the offset to the methods
instead; note that they both modify the underlying skyframe graph, so there
are side effects to the calls even if there's no direct state anymore
The intent is to migrate the relevant tests to LoadingPhaseRunnerTest (which
could also now be renamed since it's not doing a loading phase), and then
delete the TargetPatternEvaluator interface.
This depends on the previous commit that removed the last direct use of TPE
from an internal command.
PiperOrigin-RevId: 200198067
|
|
|
|
|
|
|
| |
The on-disk execRoot/blaze-out is now off limits to ActionFS.
RELNOTES: None
PiperOrigin-RevId: 200080287
|
|
|
|
|
|
|
|
| |
Derived artifacts' owners are important because they are used to determine the artifact's generating action. Source artifacts' owners are not used in this way, so I left them alone.
This allows us to get rid of most uses of ArtifactSkyKey. We may be able to delete it entirely in a follow-up.
PiperOrigin-RevId: 199836436
|
|
|
|
| |
PiperOrigin-RevId: 199775400
|
|
|
|
| |
PiperOrigin-RevId: 199732415
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Report failures in TargetPatternFunction, rather than in its callers. Since we
can't distinguish between keep_going and nokeep_going modes, we otherwise end
up double-reporting errors. In the particular case that's covered by the
build_event_stream_test.sh, we end up reporting the same target pattern as both
skipped and failed.
Unfortunately, this means we cannot report whether the target pattern was
skipped or failed, so the pattern_skipped event is now unused (if we agree that
this is acceptable, I'll remove the corresponding infrastructure).
PiperOrigin-RevId: 199593700
|
|
|
|
|
|
| |
(minor) ActionFS now implements MetadataProvider.getInput
PiperOrigin-RevId: 199575194
|
|
|
|
|
|
| |
This simplifies the code quite a bit.
PiperOrigin-RevId: 199427943
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
RuleClass.Builder now allows authors to specify whether a rule's targets
can add additional constraints on the execution platform, and to declare
additional constraints for all targets of that rule.
Targets which support this now have an attribute,
"exec_compatible_with", which supports specifying additional constraints
that the execution platform used must match.
This attribute is non-configurable. It will only affect execution
platforms used during toolchain resolution.
Part of #5217.
Change-Id: Id2400dbf869a00aa2be3e3d2f085c2850cd6dc00
Closes #5227.
Change-Id: If7d55f08f7f44bc7d7f6dfec86a3e6bcd68574b9
PiperOrigin-RevId: 199326255
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
way to remap references to repositories within an external repository by another name. This CL only adds the mappings to the Package object, but it does not actually enable the reassignments.
Example usage (in a WORKSPACE file):
local_repository(
name = ?a?,
path = ?../a?,
repo_mapping = {?@x? : ?@y?}
)
This change also creates a new SkyKey which represents the mappings. This is to prevent all packages from depending on the external package, and instead depending just on the mappings. i.e. a change to the WORKSPACE file that does not touch the mappings shouldn't cause a reload of the package.
RELNOTES:None
PiperOrigin-RevId: 199187963
|
|
|
|
|
|
| |
needed: we can get the ConfiguredTargetKey directly from the TargetCompletionKey. Since that was the only use of the actual value in EvaluationProgressReceiver#evaluated, remove it, instead just provide a boolean enum that gives whether or not evaluation succeeded.
PiperOrigin-RevId: 199178047
|
|
|
|
|
|
|
| |
This unclashes with the incoming ConfigurationTransition.apply
method described in https://docs.google.com/document/d/1_UJKmAQ9EE8i3Pl0il3YLTYr-Q9EKYYyLatt2zohfyM/edit#heading=h.96gongkwg852.
PiperOrigin-RevId: 197769784
|
|
|
|
|
|
|
|
|
| |
Extracts a class, InputArtifactData to hold the input data instead of using a raw map. This provides the flexibility needed to support both ActionFS and existing code so ActionFS does not need to rekey the input data.
Uses the smaller, getDeclaredIncludeSrcs instead of getAllowedDerivedInputs
when possible for staging optional inputs in ActionFS.
PiperOrigin-RevId: 196736703
|
|
|
|
|
|
| |
Also pass the GraphInconsistencyReciever into SkyframeExecutor as a parameter.
PiperOrigin-RevId: 196716642
|
|
|
|
|
|
| |
CommonCommandOptions to better facilitate certain kinds of benchmarking.
PiperOrigin-RevId: 196695342
|
|
|
|
|
|
|
|
|
| |
Use update() and other methods which properly respect the trimming
transition over those which don't. Avoid using the target configuration
if the configuration actually used by a target is available.
RELNOTES: None.
PiperOrigin-RevId: 196588405
|
|
|
|
| |
PiperOrigin-RevId: 195973862
|
|
|
|
| |
PiperOrigin-RevId: 194998981
|
|
|
|
|
|
|
|
|
|
| |
node's in-progress data should be forgotten, and its evaluation should be restarted from scratch, as if it were freshly created/dirtied. To guard against this happening unexpectedly, any such events are passed to a GraphInconsistencyReceiver, which can verify that the SkyFunction is behaving properly.
This is the first change in a series to permit action rewinding when it is discovered that a previously generated input file is no longer available. When an action detects that one of its inputs is unusable, it can return this sentinel value, causing it to be re-evaluated from scratch. Follow-up changes will make the node corresponding to the input, and the node corresponding to the action that generated the input, dirty when this happens, causing the upstream action to be re-run, regenerating the desired input.
Currently works for builds that do not keep edges, although follow-ups may make this possible for all builds.
PiperOrigin-RevId: 194863097
|
|
|
|
|
|
|
|
| |
Allows subclasses to use different strategies for injecting Skylark
semantics.
RELNOTES: None.
PiperOrigin-RevId: 194546199
|
|
|
|
| |
PiperOrigin-RevId: 194387085
|