| Commit message (Collapse) | Author | Age |
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 208646319
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This works around the following error:
java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at com.google.devtools.build.lib.profiler.Profiler$BinaryFormatWriter.run(Profiler.java:949)
at java.lang.Thread.run(Thread.java:748)
JDK 9 introduced an overloaded methods with covariant return type.
Closes #5886.
PiperOrigin-RevId: 208642556
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 208509424
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 208462186
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 208195833
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
output was written all in one line. However, it is really convenient to be able
to grep and count over the file (or generally be able to open it in an editor).
This change is a bit hacky as just using setIndent makes the file completely
expanded with one key-value pair per line, which is also not ideal. With this
change, the format is:
[
{ <entry> },
{ <entry> },
...
{ <entry> }
]
RELNOTES: None
PiperOrigin-RevId: 206758496
|
|
|
|
| |
PiperOrigin-RevId: 206729076
|
|
|
|
|
|
|
| |
chrome://tracing is able to load gzipped profiles out of the box.
RELNOTES: None
PiperOrigin-RevId: 206308018
|
|
|
|
|
|
|
| |
This significantly reduces the size of the resulting profiles. Do we need the
events that I remove? What do we do with the data? Who knows?
PiperOrigin-RevId: 206153368
|
|
|
|
| |
PiperOrigin-RevId: 205985818
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Follow-up to commit 09d20311d982606093ed881d779bb05a5ee70ed3.
Use try-with-resources to ensure Writer objects
are closed eagerly.
Eagerly closing Writers avoids hanging on to
file handles until the garbage collector finalizes
the object, meaning Bazel on Windows (and
other processes) can delete or mutate these files.
Hopefully this avoids intermittent file deletion
errors that sometimes occur on Windows.
See https://github.com/bazelbuild/bazel/issues/5512
RELNOTES: none
PiperOrigin-RevId: 203934471
|
|
|
|
|
|
|
| |
Also add preconditions check.
RELNOTES: None.
PiperOrigin-RevId: 203731929
|
|
|
|
|
|
|
| |
PiperOrigin-RevId: 203300374
Change-Id: Iaa47f870ab2e0cad40a202aad2c7f9430f73c856
PiperOrigin-RevId: 203407392
|
|
|
|
|
|
|
| |
same actions.
RELNOTES: None
PiperOrigin-RevId: 203120069
|
|
|
|
|
|
|
| |
Also correct for buggy profiles written previously.
RELNOTES: None.
PiperOrigin-RevId: 202920255
|
|
|
|
|
|
| |
This reduces our reliance on ProfilerTask.
PiperOrigin-RevId: 200538769
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add a --experimental_generate_json_trace_profile option that puts a file into
the output base (or uses --profile if set).
There are still a lot of problems with this.
- unexplained holes
- way too many threads
- nonsensical event titles
- too many detail events, too little overview
- it may also cause unnecessary load
- it silently overwrites the existing file on subsequent invocations
The format is documented here: goo.gl/oMZPLh
PiperOrigin-RevId: 200259431
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- move the save method to an inner class
- don't use a timer, use a blocking queue instead
- add a format enum (in anticipation of adding a json output format)
- update the test to use an in memory buffer, and avoid FoundationTestCase
Compared to the original https://github.com/bazelbuild/bazel/commit/15b8c259db111012b4642287172cb4d1d82151f3, it contains these changes:
- Make it so we don't create a queue if we are not going to write any
data! The queue is now owned by the writer, and if there is no writer, there
is no queue.
This was causing a memory regression because slowest task profiling is
enabled by default, in which case the profiler is started with no output
file. In that case, there's no thread that is emptying the queue, but the
queue was still created by default.
- add additional tests for slowest task and histogram handling; these also
provide coverage for the case where the profiler is started without an output
stream
- move all the writer thread handling into the inner class
- make writer access thread-safe
- add a bunch of documentation
PiperOrigin-RevId: 200212978
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Breaks internal performance tests.
PiperOrigin-RevId: 200103033
|
|
|
|
|
|
|
|
|
| |
- move the save method to an inner class
- don't use a timer, use a blocking queue instead
- add a format enum (in anticipation of adding a json output format)
- update the test to use an in memory buffer, and avoid FoundationTestCase
PiperOrigin-RevId: 200065404
|
|
|
|
|
|
| |
- migrate all startTask/completeTask pairs to the new API
PiperOrigin-RevId: 200038703
|
|
|
|
| |
PiperOrigin-RevId: 199849102
|
|
|
|
|
|
|
|
| |
With this change, the parser explicitly asks the lexer to give the next token.
To avoid changing the lexer too much, the tokenize() method populates a queue (it may add multiple tokens at the same time). While this reduces the peak memory usage, further work is needed to actually improve the performance.
RELNOTES: None.
PiperOrigin-RevId: 197576326
|
|
|
|
|
|
|
|
|
| |
Extracts a class, InputArtifactData to hold the input data instead of using a raw map. This provides the flexibility needed to support both ActionFS and existing code so ActionFS does not need to rekey the input data.
Uses the smaller, getDeclaredIncludeSrcs instead of getAllowedDerivedInputs
when possible for staging optional inputs in ActionFS.
PiperOrigin-RevId: 196736703
|
|
|
|
| |
PiperOrigin-RevId: 196266567
|
|
|
|
| |
PiperOrigin-RevId: 195141891
|
|
|
|
| |
PiperOrigin-RevId: 195040539
|
|
|
|
|
|
|
|
| |
Looks like a typo resulted in getSlowestTasks accumulating way more than it
should have and we were missing the test coverage to catch it.
RELNOTES: None
PiperOrigin-RevId: 194169355
|
|
|
|
|
|
| |
Profiling can hold onto objects for the duration of the build, and some of those objects may be temporary that should not be persisted. In particular, UnixGlob and its inner classes should not outlive loading and analysis. For the most part, care was taken in this CL to only use strings that required no additional construction, mainly to minimize garbage (retaining references to newly created strings is not as great a concern since only the strings corresponding to the slowest K tasks are retained, for some relatively small values of K). Action descriptions for actually executing actions are eagerly expanded because that work is minimal compared to the work of actually executing an action.
PiperOrigin-RevId: 191251488
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Remove Optional<> where it's not needed. It's nice for return values, but IMHO it was overused in this code (e.g. Optional<List<X>> is an anti-pattern, as the list itself can already signal that it is empty).
- Use Bazel's own Path class when dealing with paths, not String or java.io.File.
- Move LinuxSandboxUtil into the "sandbox" package.
- Remove dead code and unused fields.
- Migrate deprecated VFS method calls to their replacements.
- Fix a bug in ExecutionStatistics where a FileInputStream was not closed.
Closes #4868.
PiperOrigin-RevId: 190217476
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 190073818
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 187370833
|
|
|
|
|
|
| |
the end of the build in an effort to get an accurate measurement of used memory.
PiperOrigin-RevId: 187337487
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 186658512
|
|
|
|
|
|
|
| |
We call Map#put in Map#computeIfAbsent. No idea how this ever worked.
RELNOTES: None
PiperOrigin-RevId: 185876437
|
|
|
|
|
|
| |
Exports makes it hard to reason statically about which targets actually depend on any given target. It also tends to turn targets into buckets with everything, creating unnecessary edges.
PiperOrigin-RevId: 175080282
|
|
|
|
|
|
|
|
| |
Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own.
This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually.
PiperOrigin-RevId: 175033526
|
|
|
|
|
|
| |
This requires a fairly large amount of changes to fundamental objects like BlazeRuntime, Executor, and so on, as well as changing a lot of test code to thread the file system through. I expect future CLs to be much smaller.
PiperOrigin-RevId: 173678144
|
|
|
|
|
|
|
|
|
|
|
| |
This adds two dump command, bazel dump --rules and bazel dump --skylark_memory.
dump --rules outputs a summary of the count, action count, and memory consumption of each rule and aspect class.
dump --skylark_memory outputs a pprof-compatible file with all Skylark analysis allocations. Users can then use pprof as per normal to analyse their builds.
RELNOTES: Add memory profiler.
PiperOrigin-RevId: 172558600
|
|
|
|
|
|
|
|
|
|
|
| |
This simple system allows blaze developers to insert instrumentations in particular methods that they want to know:
1. How often are they called?
2. From which call sites are they called, with full call stack
The output is a pprof file that can then be analysed offline.
PiperOrigin-RevId: 172128440
|
|
|
|
|
|
| |
Closes #3697.
PiperOrigin-RevId: 168353082
|
|
|
|
|
|
|
|
|
| |
Split collect, concurrent, vfs, windows into package-level BUILD files.
Move clock classes out of "util", into their own Java package.
Move CompactHashSet into its own Java package to break a dependency cycle.
Give nestedset and inmemoryfs their own package-level BUILD files.
PiperOrigin-RevId: 167702127
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 167574104
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 167505493
|
|
|
|
|
|
|
|
|
|
| |
- Move ProfilerInfo into a subpackage (it's not necessary for profiling, just for analyzing a profile).
- Make some fields in Profiler public for ProfileInfo.
- Mark Profiler as ThreadSafe; there's no cyclic dependency here.
This is based on ulfjack's microbazel patch series: https://github.com/ulfjack/bazel/commit/44553fcac0fc876784d8f48c2e577d8c999712de
PiperOrigin-RevId: 167121952
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 160951982
|
|
|
|
|
|
|
| |
Fixes fallout from https://github.com/bazelbuild/bazel/commit/fda985b069ed4cc1966a6ced5743c396f91ac688.
RELNOTES: none
PiperOrigin-RevId: 160626838
|
|
|
|
|
|
|
| |
With a few manual fixes for readability.
RELNOTES: None.
PiperOrigin-RevId: 160582556
|
|
|
|
|
|
|
|
|
| |
the currently defined hash function for blobs. Some refactoring. Adding an option to set the hash function in the remote worker, defaulting to the current behavior (unfortunately it is a build option, have not found a clean way to specify it at runtime).
BUG=62622420
TESTED=remote worker
RELNOTES: none
PiperOrigin-RevId: 159473116
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 147734029
MOS_MIGRATED_REVID=147734029
|