| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Roll forward of directory name change
*** Original change description ***
Automated g4 rollback of commit 1d9e1ac90197b1d3d7b137ba3c1ada67bb9ba31b.
*** Reason for rollback ***
Breaks //src/test/shell/integration:force_delete_output_test
*** Original change description ***
Symlink output directories to the correct directory name
If the workspace directory is /path/to/my/proj and the name in the WORKSPACE
file is "floop", this will symlink the output directories to
output_base/execroot/floop instead of output_base/execroot/proj.
More prep for #1262, fixes #1681.
PiperOrigin-RevId: 156892980
|
|
|
|
|
|
|
|
|
|
| |
In preparation to support multi-configuration builds, provide infrastructure
allowing build events to reference BuildConfigurations. The streaming mechanism
will ensure that build configurations are introduced in the stream before being
referenced for the first time.
Change-Id: I6b96fbebc76a05eff4f75a07e8a9cfbcd57f9c22
PiperOrigin-RevId: 155368666
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Filter out conflicting flag policies for applicable commands. Only enforce the filtered, applicable policy.
*** Original change description ***
Automated g4 rollback of commit aa7f9307636d38cbb93a03acac8f4c59adfa0ee8.
*** Reason for rollback ***
Broke --experimental_inmemory_dotd_files
*** Original change description ***
Filter out conflicting flag policies before invocation policy enforcement.
This is to minimize the likelihood of obscure policy conflict. Now, the last policy on a flag (after policy expansion) will be the only one in the "canonical" invocation policy. There should be no reason for explicitly setting multiple policies on a single flag, but if an expansion flag is policy'd and one of its children has a more specific policy on it, make sure that the policy on the child flag is after the policy on the expansion flag.
Note that this restriction (only the last policy gets applied) also applies for repeatable flags. Make sure all values being set to a repeatable flag are set in a single SetValue operation, with multiple flagValues set.
PiperOrigin-RevId: 154969189
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Extend the build-event streamer to also report about stdout/stderr,
if provided. This information is reported in the progress events. At
the moment, we only report stdout/stderr in progress events we send
anyway, but the interface is generic enough that we could add time-based
reporting later, if needed. Also note, that at the end of the build, we
report the final progress event, so that all stdout/stderr generated before
the build-complete event get also reported in the build-event protocol.
Change-Id: If5dbd59c151edbce02d0a9b2e5938b63c0a5dc58
PiperOrigin-RevId: 154811110
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Broke --experimental_inmemory_dotd_files
RELNOTES: None
PiperOrigin-RevId: 154477949
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Rollforward with fix for test flakiness. BEP transport closed
events are delivered via their own threadpool and thus might
not have been sent immediately. BuildEventStreamerTest#testSimpleStream
now waits for a bit until the event has been delivered. I ran the test
with --runs_per_test=1000 several times and had no further failures.
*** Original change description ***
Automated g4 rollback of commit 3d596d63f883fff56001ed7b2e5cf51dba45f082.
*** Reason for rollback ***
Made BuildEventStreamerTest#testSimpleStream 3% flaky based on --runs_per_test=1000.
RELNOTES: None
PiperOrigin-RevId: 154170833
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Made BuildEventStreamerTest#testSimpleStream 3% flaky based on --runs_per_test=1000.
RELNOTES: None
PiperOrigin-RevId: 154090559
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The BEP (build event protocol) upload may at times take longer than the build
itself. This is especially true for cached builds, where there is little build
work, but the protocol still needs to be uploaded. In this case the bazel UI
will now display that it's waiting for BEP upload, both in the current and the
new experimental UI (--experimental_ui).
When executing a run command, blaze waits for the BEP upload to finish before
it runs the target.
Major Modifications:
- The BuildEventTransport interface now also has a name() method that returns
a string. When waiting for a transport to finish the BEP upload, this string
is displayed in the UI.
- The BuildEventStreamer now emits two new events
AnnounceBuildEventTransportsEvent and BuildEventTransportClosed on the event
bus. This is how the experimental UI is informed about BEP upload progress.
RELNOTES: None
PiperOrigin-RevId: 154052401
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is to minimize the likelihood of obscure policy conflict. Now, the last policy on
a flag (after policy expansion) will be the only one in the "canonical" invocation
policy. There should be no reason for explicitly setting multiple policies on a single
flag, but if an expansion flag is policy'd and one of its children has a more specific
policy on it, make sure that the policy on the child flag is after the policy on the
expansion flag.
Note that this restriction (only the last policy gets applied) also applies for repeatable flags. Make sure all values being set to a repeatable flag are set in a single SetValue operation, with multiple flagValues set.
PiperOrigin-RevId: 153584034
|
|
|
|
|
|
|
|
| |
There's no reason an expansion flag should not expand to multiple values for a repeatable flag (a flag with allowMultiple set to true.) If this expansion flag is set in a SetValue policy, group its repeatable subflags into a single SetValue per subflag.
For an overridable SetValue policy on an expansion, any repeatable flag that it expands to should append its value, and not override the user's original values.
PiperOrigin-RevId: 153233784
|
|
|
|
|
|
| |
By the time the invocation policy gets enforced, the expansion flag should have been
expanded, and policy should act as on any other expansion flag.
PiperOrigin-RevId: 152692831
|
|
|
|
|
|
|
|
|
|
| |
Extend the functionality of the BuildEventStreamer to report those parts
of NestedSets of Artifacts not reported earlier. In this way, duplicate
reporting can be avoided, without the events themselves having to know
which artifacts are known already.
Change-Id: Ia959c28c440301860eac57ea5d9a712c0d49ebdf
PiperOrigin-RevId: 152497672
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Better specify the BuildEventTransport interface. Besides clarifying threading
and blocking issues, this change also clarifies error handling/reporting.
After several discussions we concluded that the BuildEventTransport interface
should not provide error reporting / handling facilities, as there is not much
bazel could do with this information. Instead, a transport may decide for
itself if an error is fatal and abort the build or if an error should be logged
to the user's terminal or if it should be ignored.
Furthermore, changing the close() method lays the groundwork for an upcoming
change that will report the transport shutdown status to the user command
line.
PiperOrigin-RevId: 152408938
|
|
|
|
|
|
|
|
|
|
|
|
| |
'create' method.
This paves the way for changing PathFragment to e.g. an abstract class with multiple subclasses. This way we can split out the windows-specific stuff into one of these concrete classes, making the code more readable and also saving memory (since the shallow heap size of the NonWindowsPathFragment subclass will hopefully be smaller than that of the current PathFragment).
This also lets us pursue gc churn optimizations. We can now do interning in PathFragment#create and can also get rid of unnecessary intermediate PathFragment allocations.
RELNOTES: None
PiperOrigin-RevId: 152145768
|
|
|
|
|
|
|
|
|
|
|
| |
Change the BuildEvent interface to accept a generic class of converters.
In this way, we won't have to change it again in the future, once more
converters are needed. In fact, a new converter is needed right now (will
be added in a follow-up patch) to allow build events to know the name of
named artifact groups already reported in the stream.
Change-Id: Ibb32ea5fff361e21bcf2d34818d8351a1da7a2e3
PiperOrigin-RevId: 152131870
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Breaks //src/test/shell/integration:force_delete_output_test
*** Original change description ***
Symlink output directories to the correct directory name
If the workspace directory is /path/to/my/proj and the name in the WORKSPACE
file is "floop", this will symlink the output directories to
output_base/execroot/floop instead of output_base/execroot/proj.
More prep for #1262, fixes #1681.
PiperOrigin-RevId: 152126545
|
|
|
|
|
|
|
|
|
|
|
|
| |
--incompatible_* flags
Note that if a developer adds a poorly-formatted incompatible change @Option, constructing an OptionsParser will now fail with an unchecked exception. This can cause some unit tests to fail in unexpected ways, but the developer should see an appropriate error message when the server starts up.
To be added: A separate integration test that ensures the expansions of --all_incompatible_changes don't clobber each other.
RELNOTES: None
PiperOrigin-RevId: 151858287
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
--noexperimental_enable_critical_path_profiling flags are all specified, then Bazel will delete Actions from ActionLookupValues as they are executed in order to save memory.
Because an already-run action may output an artifact that is only requested later in the build, we need to maintain a way for the artifact to look up the action. But in most cases we don't need to keep the action itself, just its output metadata.
Some actions unfortunately are needed post-execution, and so we special-case them.
Also includes dependency change with description:
Move action out of key. This keeps action references from polluting the graph -- actions are just stored in one SkyValue, instead of being present in SkyKeys.
This does mean additional memory used: we have a separate ActionLookupData object per Action executed. That may reach ~24M for million-action builds.
PiperOrigin-RevId: 151756383
|
|
|
|
|
|
|
|
| |
For SetValue and UseDefault policies on expansion flags or flags with implicitRequirements, expand the policy into policy for each of its sub-flags. For SetValue, this addresses the issue with policies on expansion flags with overridable=true not actually letting user flags overrride the expansion. For UseDefault, this formalizes the behavior where UseDefault will wipe all user-provided flags that expand from a banned expansion flag, and will allow later work to guarantee that a later policy can override the expansion policy's subflags.
Since expansion flags do not have value, break if the invocation policy uses AllowValue or DisallowValue on an expansion flag.
PiperOrigin-RevId: 151718539
|
|
|
|
|
|
|
|
|
|
| |
If the workspace directory is /path/to/my/proj and the name in the WORKSPACE
file is "floop", this will symlink the output directories to
output_base/execroot/floop instead of output_base/execroot/proj.
More prep for #1262, fixes #1681.
PiperOrigin-RevId: 151712384
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It's possible for cases to happen where the BlazeCommandDispatcher
fails before starting the WaitForCompletionCommand. In these cases,
the test's command state handoff will never take place because no
command state will be created to be handed off (obviously).
This means that runIn will wait forever.
Instead, if the command completes and the command state has still
not been handed off, set an exception into the command state so
that the test will fail without further troubles.
RELNOTES: None.
PiperOrigin-RevId: 151495494
|
|
|
|
|
|
| |
The encapsulation was off, since the format in which we accept policy has nothing to do with how we then enforce the policy.
PiperOrigin-RevId: 151471747
|
|
|
|
|
|
|
|
| |
It was large and unwieldy - keep common logic in the original file but split out the tests into groups by invocation policy operation type.
--
PiperOrigin-RevId: 150759733
MOS_MIGRATED_REVID=150759733
|
|
|
|
|
|
|
|
| |
Remove magic constants and convert to Truth.
--
PiperOrigin-RevId: 150692720
MOS_MIGRATED_REVID=150692720
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Always initialize the set of postedEvents. Also, record the initial event
as posted. In this way, the build event streamer can also handle events that
should be posted after the BuildStartedEvent but are told the streamer about
earlier.
--
Change-Id: Ic796f2434434819e6c62633755411527a9dcb7e0
Reviewed-on: https://cr.bazel.build/9451
PiperOrigin-RevId: 150620387
MOS_MIGRATED_REVID=150620387
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 150051360
MOS_MIGRATED_REVID=150051360
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 149418372
MOS_MIGRATED_REVID=149418372
|
|
|
|
|
|
|
|
|
|
|
|
| |
In the experimental UI, to be able to report appropriately about ongoing
downloads, we need to track their state, as updated by the DownloadProgressEvents.
Do so. Also report on ongoing downloads in the progress bar.
--
Change-Id: I668e963cd6da85ec598b23724066d366d465271f
Reviewed-on: https://cr.bazel.build/9114
PiperOrigin-RevId: 148899297
MOS_MIGRATED_REVID=148899297
|
|
|
|
|
|
|
|
|
|
|
|
| |
The build event protocol now emits a BuildFinished event at the end of a build. The event is a child of the BuildStarted event.
The code changes were larger than expected, due to some refactoring in BuildEventStreamer. This was necessary as the BuildCompleteEvent now implements the BuildEvent interface and Guava's EventBus always invokes the most specialized subscriber method. Thus, the buildEvent(BuildEvent) and buildCompleted(BuildCompletedEvent) methods had to be merged.
--
Change-Id: Id0c2bd7220dc8ce03128b7126587e212ee8ce836
Reviewed-on: https://cr.bazel.build/9053
PiperOrigin-RevId: 148788582
MOS_MIGRATED_REVID=148788582
|
|
|
|
|
|
|
|
|
|
|
| |
Bazel-created files (like log files of test runs) are internally reported
as Paths. However, this is not always the most useful representation of the
location of that artifact for a consumer of build events. Therefore, support
a mapping of paths to more useful URIs.
--
PiperOrigin-RevId: 144843525
MOS_MIGRATED_REVID=144843525
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
which have converters that return lists. The problem was that the policy might,
say, disallow values ["a", "b"], and a flag --foo might have a converter which
takes a string and splits it by commas to produce a list. The options parser
would apply the converter to --foo=a,b to produce ["a", "b"], but invocation
policy would compare each element of the policy to the list itself, which will
never work.
--
PiperOrigin-RevId: 142297177
MOS_MIGRATED_REVID=142297177
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If an Aspect registered an action that an extra-action is
shadowing, its name is used when creating the extra-action's ID and
name.
Since recently, an aspect can see other aspects applied to the same
target. This CL record the names of other aspects applied to the target
as well, disambiguating the action owners.
--
PiperOrigin-RevId: 142264153
MOS_MIGRATED_REVID=142264153
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In the build event protocol, we promise to respect certain ordering
constraints that come naturally by causality anyway (e.g., the root
cause of a failure has to come before the failure). However, in the
way the call structure is organized, events might occur on the event
bus in wrong order. So allow events to specify that order and make the
streamer honor it.
--
Change-Id: I1fbe9b93681f0ddfa35d9d00d5d1b02103da38a5
Reviewed-on: https://bazel-review.googlesource.com/#/c/7370
MOS_MIGRATED_REVID=139774946
|
|
|
|
|
|
|
| |
its buffer.
--
MOS_MIGRATED_REVID=139771073
|
|
|
|
|
|
|
|
|
| |
to action_listener() rules.
RELNOTES: Extra actions now contain aspect-related information.
--
MOS_MIGRATED_REVID=138832922
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=138680612
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
delimited).
Adds --experimental_build_event_binary_file option that enables varint delimited proto loggging to the specified file path
Adds varint delimited BuildEventStreamTransport and BuildEventStreamerModule
Adds BuildEventStreamerModule for configuring and setting up BuildEventStreamer and its associated BuildEventTransports.
Adds BuildEventTransportFactory which creates a Set of transports from command options.
Moves BuildEventStreamer configuration from BlazeCommandDispatcher and BuildEventStreamerModule
--
Change-Id: If71f2b58654879c2509206da47e6d1a846bf397f
Reviewed-on: https://bazel-review.googlesource.com/#/c/7010/
MOS_MIGRATED_REVID=138073726
|
|
|
|
|
|
|
|
|
| |
a value to use instead of the given flag value if it is disallowed by the
policy. This is a generalization of the existing new_default_value behavior to
cover any value of the flag, not just the default value if the flag is unset.
--
MOS_MIGRATED_REVID=137104944
|
|
|
|
|
|
|
| |
Ideally, stdout would be line buffered, too, except that we sometimes pump large amounts of binary data through it, so line buffering would result in performance loss.
--
MOS_MIGRATED_REVID=135237450
|
|
|
|
|
|
|
| |
--
Change-Id: If2cd06ba19e1035275f30c445a99af7a7c81e1d8
Reviewed-on: https://bazel-review.googlesource.com/#/c/6430
MOS_MIGRATED_REVID=135210119
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is the first step on a journey toward allowing commands to AbruptExit
wherever they please, similar to how the user can press Ctrl+C at any time
and we (should) bail out as fast as we can.
By interrupting the command's main thread, we at least offer the command
the ability to see that an error requiring a bail has happened, and it should
trigger at potentially more locations, rather than just between phases.
--
MOS_MIGRATED_REVID=135152330
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The BuildEventStreamer will listen for BuildEvents and stream them
to the provided transports. It also ensures events are properly chained:
for unsolicited events, it will add progress events to chain them and
at the end of a build it closes all announced but not produced events
as aborted.
--
Change-Id: I623b582657573fe1288821c96f084e0ab0bca4d4
Reviewed-on: https://bazel-review.googlesource.com/#/c/6275
MOS_MIGRATED_REVID=134787541
|
|
|
|
|
|
|
| |
Require all callers to call initWorkspace after BlazeRuntime construction.
--
MOS_MIGRATED_REVID=129842292
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is done in preparation for allowing multiple workspaces / commands in a
single server. That requires changes to the module API, which currently
hard-codes a 1:1 correspondence between workspaces and servers. Note how
BlazeDirectories exists even when it runs outside a workspace.
It's not ideal that the output base is in ServerDirectories, and the
BlazeRuntime creation also still requires a BlazeDirectories instance.
--
MOS_MIGRATED_REVID=129736613
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When requested to produce a suffix of a string of a string
of a given length, gracefully handle the case where the requested
length is negative---simply return the empty string in
this case. While there, mark this static method as such; also
increase visibility to default as it is generally useful and
should be tested as an interface of this class.
--
Change-Id: I821966f7ba3828809bc6d000358803c131740ec9
Reviewed-on: https://bazel-review.googlesource.com/#/c/4223
MOS_MIGRATED_REVID=129080284
|
|
|
|
|
|
|
| |
This change is similar to a previous change that introduced WorkspaceBuilder.
--
MOS_MIGRATED_REVID=126799657
|
|
|
|
|
|
|
|
|
| |
...so that it can be used by the main code base.
--
Change-Id: Ibf59d74ab1b0cca9bd9406bb76ee6e71189653bf
Reviewed-on: https://bazel-review.googlesource.com/#/c/3903
MOS_MIGRATED_REVID=125950656
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, the experimental UI always shows the 3 oldest, still running actions.
While this seems a reasonable default, some users requested to be able to change
the number of actions shown. Hence replace the hard-coded value by a flag.
While there, also fix an off-by-one error in deciding when to put the ellipsis
symbol.
--
Change-Id: I037d208360fa1d3f100c99ab1ab1f5fc140138ac
Reviewed-on: https://bazel-review.googlesource.com/#/c/3811
MOS_MIGRATED_REVID=125427168
|
|
|
|
|
|
|
|
|
|
| |
In the experimental UI, for the running actions also report their strategy.
This will give a more complete picture of what Bazel is currently doing.
--
Change-Id: I9553c952ed494e0db225b2a1ae5e8eba00f68617
Reviewed-on: https://bazel-review.googlesource.com/#/c/3820
MOS_MIGRATED_REVID=125162808
|
|
|
|
|
|
|
|
|
|
|
| |
The experimental UI also keeps track, in the progress bar, of the last test
that completed. When using curses, use colors to indicate whether the test
passed or not.
--
Change-Id: Iaa01a773c3bbf534692ed21dd420596cb63e2585
Reviewed-on: https://bazel-review.googlesource.com/#/c/3752
MOS_MIGRATED_REVID=123871492
|