| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- move the save method to an inner class
- don't use a timer, use a blocking queue instead
- add a format enum (in anticipation of adding a json output format)
- update the test to use an in memory buffer, and avoid FoundationTestCase
Compared to the original https://github.com/bazelbuild/bazel/commit/15b8c259db111012b4642287172cb4d1d82151f3, it contains these changes:
- Make it so we don't create a queue if we are not going to write any
data! The queue is now owned by the writer, and if there is no writer, there
is no queue.
This was causing a memory regression because slowest task profiling is
enabled by default, in which case the profiler is started with no output
file. In that case, there's no thread that is emptying the queue, but the
queue was still created by default.
- add additional tests for slowest task and histogram handling; these also
provide coverage for the case where the profiler is started without an output
stream
- move all the writer thread handling into the inner class
- make writer access thread-safe
- add a bunch of documentation
PiperOrigin-RevId: 200212978
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Breaks internal performance tests.
PiperOrigin-RevId: 200103033
|
|
|
|
|
|
|
|
|
| |
- move the save method to an inner class
- don't use a timer, use a blocking queue instead
- add a format enum (in anticipation of adding a json output format)
- update the test to use an in memory buffer, and avoid FoundationTestCase
PiperOrigin-RevId: 200065404
|
|
|
|
|
|
| |
- migrate all startTask/completeTask pairs to the new API
PiperOrigin-RevId: 200038703
|
|
|
|
| |
PiperOrigin-RevId: 199849102
|
|
|
|
|
|
|
|
| |
With this change, the parser explicitly asks the lexer to give the next token.
To avoid changing the lexer too much, the tokenize() method populates a queue (it may add multiple tokens at the same time). While this reduces the peak memory usage, further work is needed to actually improve the performance.
RELNOTES: None.
PiperOrigin-RevId: 197576326
|
|
|
|
|
|
|
|
|
| |
Extracts a class, InputArtifactData to hold the input data instead of using a raw map. This provides the flexibility needed to support both ActionFS and existing code so ActionFS does not need to rekey the input data.
Uses the smaller, getDeclaredIncludeSrcs instead of getAllowedDerivedInputs
when possible for staging optional inputs in ActionFS.
PiperOrigin-RevId: 196736703
|
|
|
|
| |
PiperOrigin-RevId: 196266567
|
|
|
|
| |
PiperOrigin-RevId: 195141891
|
|
|
|
| |
PiperOrigin-RevId: 195040539
|
|
|
|
|
|
|
|
| |
Looks like a typo resulted in getSlowestTasks accumulating way more than it
should have and we were missing the test coverage to catch it.
RELNOTES: None
PiperOrigin-RevId: 194169355
|
|
|
|
|
|
| |
Profiling can hold onto objects for the duration of the build, and some of those objects may be temporary that should not be persisted. In particular, UnixGlob and its inner classes should not outlive loading and analysis. For the most part, care was taken in this CL to only use strings that required no additional construction, mainly to minimize garbage (retaining references to newly created strings is not as great a concern since only the strings corresponding to the slowest K tasks are retained, for some relatively small values of K). Action descriptions for actually executing actions are eagerly expanded because that work is minimal compared to the work of actually executing an action.
PiperOrigin-RevId: 191251488
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Remove Optional<> where it's not needed. It's nice for return values, but IMHO it was overused in this code (e.g. Optional<List<X>> is an anti-pattern, as the list itself can already signal that it is empty).
- Use Bazel's own Path class when dealing with paths, not String or java.io.File.
- Move LinuxSandboxUtil into the "sandbox" package.
- Remove dead code and unused fields.
- Migrate deprecated VFS method calls to their replacements.
- Fix a bug in ExecutionStatistics where a FileInputStream was not closed.
Closes #4868.
PiperOrigin-RevId: 190217476
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 190073818
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 187370833
|
|
|
|
|
|
| |
the end of the build in an effort to get an accurate measurement of used memory.
PiperOrigin-RevId: 187337487
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 186658512
|
|
|
|
|
|
|
| |
We call Map#put in Map#computeIfAbsent. No idea how this ever worked.
RELNOTES: None
PiperOrigin-RevId: 185876437
|
|
|
|
|
|
| |
Exports makes it hard to reason statically about which targets actually depend on any given target. It also tends to turn targets into buckets with everything, creating unnecessary edges.
PiperOrigin-RevId: 175080282
|
|
|
|
|
|
|
|
| |
Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own.
This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually.
PiperOrigin-RevId: 175033526
|
|
|
|
|
|
| |
This requires a fairly large amount of changes to fundamental objects like BlazeRuntime, Executor, and so on, as well as changing a lot of test code to thread the file system through. I expect future CLs to be much smaller.
PiperOrigin-RevId: 173678144
|
|
|
|
|
|
|
|
|
|
|
| |
This adds two dump command, bazel dump --rules and bazel dump --skylark_memory.
dump --rules outputs a summary of the count, action count, and memory consumption of each rule and aspect class.
dump --skylark_memory outputs a pprof-compatible file with all Skylark analysis allocations. Users can then use pprof as per normal to analyse their builds.
RELNOTES: Add memory profiler.
PiperOrigin-RevId: 172558600
|
|
|
|
|
|
|
|
|
|
|
| |
This simple system allows blaze developers to insert instrumentations in particular methods that they want to know:
1. How often are they called?
2. From which call sites are they called, with full call stack
The output is a pprof file that can then be analysed offline.
PiperOrigin-RevId: 172128440
|
|
|
|
|
|
| |
Closes #3697.
PiperOrigin-RevId: 168353082
|
|
|
|
|
|
|
|
|
| |
Split collect, concurrent, vfs, windows into package-level BUILD files.
Move clock classes out of "util", into their own Java package.
Move CompactHashSet into its own Java package to break a dependency cycle.
Give nestedset and inmemoryfs their own package-level BUILD files.
PiperOrigin-RevId: 167702127
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 167574104
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 167505493
|
|
|
|
|
|
|
|
|
|
| |
- Move ProfilerInfo into a subpackage (it's not necessary for profiling, just for analyzing a profile).
- Make some fields in Profiler public for ProfileInfo.
- Mark Profiler as ThreadSafe; there's no cyclic dependency here.
This is based on ulfjack's microbazel patch series: https://github.com/ulfjack/bazel/commit/44553fcac0fc876784d8f48c2e577d8c999712de
PiperOrigin-RevId: 167121952
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 160951982
|
|
|
|
|
|
|
| |
Fixes fallout from https://github.com/bazelbuild/bazel/commit/fda985b069ed4cc1966a6ced5743c396f91ac688.
RELNOTES: none
PiperOrigin-RevId: 160626838
|
|
|
|
|
|
|
| |
With a few manual fixes for readability.
RELNOTES: None.
PiperOrigin-RevId: 160582556
|
|
|
|
|
|
|
|
|
| |
the currently defined hash function for blobs. Some refactoring. Adding an option to set the hash function in the remote worker, defaulting to the current behavior (unfortunately it is a build option, have not found a clean way to specify it at runtime).
BUG=62622420
TESTED=remote worker
RELNOTES: none
PiperOrigin-RevId: 159473116
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 147734029
MOS_MIGRATED_REVID=147734029
|
|
|
|
|
|
|
|
| |
Preconditions check because stringification is lazy.
--
PiperOrigin-RevId: 145560352
MOS_MIGRATED_REVID=145560352
|
|
|
|
|
|
|
|
| |
Fixes #2325.
--
PiperOrigin-RevId: 143434786
MOS_MIGRATED_REVID=143434786
|
|
|
|
|
|
|
| |
finishes, which calls #clear (via #stop via BlazeRuntime#afterCommand), concurrently with an OOM or other asynchronous crash which invokes CommandCompleteEvent subscribers, which may try to write profiling data.
--
MOS_MIGRATED_REVID=124853610
|
|
|
|
|
|
|
|
|
| |
Also removed some unused code while I was here.
Fixes https://github.com/bazelbuild/bazel/issues/726.
--
MOS_MIGRATED_REVID=113758464
|
|
|
|
|
|
|
| |
the threshold to 100 ms for two common local actions to avoid spamming the INFO log too much. I would be happy to set the threshold even higher.
--
MOS_MIGRATED_REVID=113413914
|
|
|
|
|
|
|
| |
Reduces garbage.
--
MOS_MIGRATED_REVID=109914243
|
|
|
|
|
|
|
| |
of recording VFS_ stats we try to catalog the access base on the path type.
--
MOS_MIGRATED_REVID=108238357
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=107416625
|
|
|
|
|
|
|
|
| |
Re-adds the missing execution phase statistics which got lost in the
recent ProfileCommand refactoring.
--
MOS_MIGRATED_REVID=105340677
|
|
|
|
|
|
|
|
| |
Add the --combine option to produce a single aggregated statistics output for
multiple profile files. Outputs neither Skylark histograms nor the task chart.
--
MOS_MIGRATED_REVID=105051164
|
|
|
|
|
|
|
|
|
| |
Does not save all Tasks anymore and generates TasksStatistics on the fly.
Adds an ArrayList implementation for primitive longs to efficiently save a lot
of task duration data. Necessary when loading lots of profile files.
--
MOS_MIGRATED_REVID=104656311
|
|
|
|
|
|
|
|
| |
Does not change the default output, but allows reuse of PhaseText
without outputting those statistics.
--
MOS_MIGRATED_REVID=104653246
|
|
|
|
|
|
|
| |
omitting the Gantt-style charts of all tasks and --[no]histograms to omit Skylark histograms which require adding huge amounts of data to the output. Especially necessary when combining data from multiple profile files.
--
MOS_MIGRATED_REVID=104646190
|
|
|
|
|
|
|
|
| |
Options allow displaying some tasks in more detail, e.g. for inspecting what
exactly a Skylark user-defined function calls and how long that takes.
--
MOS_MIGRATED_REVID=104505599
|
|
|
|
|
|
|
|
|
|
|
| |
The headers were modified with
`find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'`
And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan.
The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand.
--
MOS_MIGRATED_REVID=103938715
|
|
|
|
|
|
|
| |
and enable their reuse
--
MOS_MIGRATED_REVID=103634406
|
|
|
|
|
|
|
| |
handle in HtmlCreator
--
MOS_MIGRATED_REVID=103528221
|