aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/collect
Commit message (Collapse)AuthorAge
* During NestedSet deserialization, maintain a list of known children outside ofGravatar cpeyser2018-03-26
| | | | | | | the global digestToChild map. Since digestToChild contains weak references, this is required to ensure the children are not GCed. PiperOrigin-RevId: 190476243
* Re-enable aliasing of CodedInputStream during deserialization, removed as ↵Gravatar janakr2018-03-23
| | | | | | | | part of primary codepath somewhere around https://github.com/bazelbuild/bazel/commit/bde43ec8a96a62b8fbf67ad60d5154cf121647d9 (and killed entirely in unknown commit. Force copying of a ByteString read from CodedInputStream in NestedSetCodec and persisted, since otherwise we might hang onto the entire buffer indefinitely. PiperOrigin-RevId: 190256337
* Optimistically check the Set within NestedSetVisitor.VisitedState using ↵Gravatar Googler2018-03-22
| | | | | | | | | contains before invoking the heavier add. This reduces cpu-work and contention. When blaze is invoked on a large fileset and the build is already up-to-date, this makes things about 1.9s (33%) faster. PiperOrigin-RevId: 190125803
* Allow NestedSetCodec to share members across multiple deserializations.Gravatar cpeyser2018-03-22
| | | | | | This avoids redundancy in memory when multiple NestedSets share a member PiperOrigin-RevId: 190085907
* Add behavior to NestedSetCodec to prevent it from running during testing.Gravatar cpeyser2018-03-21
| | | | PiperOrigin-RevId: 189906038
* Automated rollback of commit e6af9f9178e731a45e552b1d249ded90124945ba.Gravatar janakr2018-03-19
| | | | | | | | | | | | *** Reason for rollback *** See linked bug. *** Original change description *** Add behavior to NestedSetCodec to prevent it from running during testing. PiperOrigin-RevId: 189663863
* Add behavior to NestedSetCodec to prevent it from running during testing.Gravatar cpeyser2018-03-19
| | | | PiperOrigin-RevId: 189602622
* @AutoCodec WriteZipControlFileAction (and ImmutableIterable).Gravatar janakr2018-03-16
| | | | | | Since autocodec library is now a dependency of lib/collect, properly annotate ImmutableSharedKeyMap to boot. PiperOrigin-RevId: 189432552
* Splits the iterableMarshaller into runtime codecs.Gravatar shahan2018-03-14
| | | | | | Makes NestedSetCodec into a runtime codec instead of a Marshaller. PiperOrigin-RevId: 189110883
* Introduce an Extrema aggregator.Gravatar nharmata2018-02-28
| | | | | RELNOTES: None PiperOrigin-RevId: 187370833
* Better @AutoCodec for TransitiveInfoProviderMapImpl: avoid having to ↵Gravatar janakr2018-02-23
| | | | | | serialize offset table at the cost of some overhead reconstructing the table. Also fewer code changes, although there is a serialization-only method added as a hack. PiperOrigin-RevId: 186808832
* @AutoCodec TransitiveInfoProviderMapImpl and JavaInfo.Gravatar janakr2018-02-23
| | | | PiperOrigin-RevId: 186789569
* Adds ObjectCodecRegistry to {Des|S}erializationContext.Gravatar shahan2018-02-20
| | | | | | | | | | | | * AutoCodec now delegates to the registry. * Adds getSuperclass logic for resolving a codec. * Small cleanups for classes that break the registry. TODO after this change: * Explicit CODEC definitions are no longer needed and existing ones should be cleaned up. * POLYMORPHIC is no longer be needed and should be cleaned up. PiperOrigin-RevId: 186351845
* Move CommandLine, CommandLineItem, and ParamFileInfo from ↵Gravatar tomlu2018-02-15
| | | | | | | | | lib.analysis.actions -> lib.actions. These are fundamental types that want to sit alongside types like Spawn. RELNOTES: None PiperOrigin-RevId: 185887971
* Add a CODEC for CppCompileAction.Gravatar cpeyser2018-02-14
| | | | PiperOrigin-RevId: 185733313
* Replaces InjectingObjectCodec with dependencies threaded through ↵Gravatar shahan2018-02-13
| | | | | | (Des|S)erializationContext. PiperOrigin-RevId: 185547740
* Add context argument to ObjectCodec.{serialize,deserialize}Gravatar michajlo2018-02-11
| | | | | | | | | | Context implementations are currently empty, just doing the plumbing in this change. Once this is in we can start passing along the ObjectCodecRegistry, which will allow runtime codec resolution for classes not known at compile time. We'll also inevitably add some memoization helpers, allowing us to optimize the serialization process further. PiperOrigin-RevId: 185305674
* Add NestedSet support in AutoCodec for type parameters which have an ↵Gravatar cpeyser2018-02-05
| | | | | | InjectingObjectCodec. PiperOrigin-RevId: 184566571
* Allow CustomCommandLine's mapFn to expand each object into multiple items.Gravatar tomlu2018-02-01
| | | | | | This is needed to migrate JavaCompileAction away from CustomMultiArgv. PiperOrigin-RevId: 184136486
* Improve safety of NestedSetFingerprintCache by detecting multiple instances ↵Gravatar tomlu2018-01-31
| | | | | | | | | | | | of the same mapFn class. This code tries to add protection against the user creating new mapFn instances per-rule. This would cause the nested set cache to be computed per-rule instead of shared across rule instances, causing memory bloat and slowdowns. Since this can only happen in native code, we can get away with detecting this and crashing blaze. I think this is a better choice than silently allowing it / falling back to slow computations. The user can override this behaviour by inheriting from CommandLineItem.CapturingMapFn, in which case the user is explicitly saying they assume responsibility for the number of instances of the mapFn the application will use. PiperOrigin-RevId: 184061642
* Add memory-efficient map for storing nested set -> digest.Gravatar tomlu2018-01-31
| | | | | | | | | Instead of using ConcurrentHashMap, we use a dead-simple open addressed hash hable with a giant byte array with 16-byte slots. We then read or write fingerprints straight into and out of the array, obviating the need to generate intermediate garbage. Locking mechanism is a read-write lock. This should be faster than full synchronisation for read-heavy loads. RELNOTES: None PiperOrigin-RevId: 184019301
* Use nested set cache in key computation for CustomCommandLine.Gravatar tomlu2018-01-29
| | | | | RELNOTES: None PiperOrigin-RevId: 183727976
* Serializer implementation for NestedSetGravatar shahan2018-01-25
| | | | | | | | | | | | | Adds some logging to test helpers for size of serialized data. Jan 25, 2018 7:16:25 AM com.google.devtools.build.lib.skyframe.serialization.testutils.SerializerTester testSerializeDeserialize INFO: total serialized bytes = 70 Jan 25, 2018 7:16:25 AM com.google.devtools.build.lib.skyframe.serialization.testutils.ObjectCodecTester testSerializeDeserialize INFO: total serialized bytes = 208 Kryo output is significantly smaller. PiperOrigin-RevId: 183300353
* Add CommandLineItem interface.Gravatar tomlu2018-01-25
| | | | | | | | This interface makes it clearer in the type system exactly how items that go into a CustomCommandLine are turned into strings. It is a preparatory change to allow command line fingerprints to be more cheaply calculated, but it is valuable in itself from a code quality standpoint. PiperOrigin-RevId: 183274022
* Avoid expansion of NestedSets in MiddlemanFactory.Gravatar Benjamin Peterson2017-12-11
| | | | | | | | | | | | MiddlemanFactory wants to check if the middleman inputs set is empty or singleton. A NestedSet can answer both queries efficiently if the right APIs are used. My ultimate goal here is to avoid the expansion of runfiles artifacts nested sets until execution. Change-Id: I29a269df757ef41b1410bbb492cf24c926df6114 PiperOrigin-RevId: 178600943
* Add ActionKeyContext to Action#getKey.Gravatar tomlu2017-11-29
| | | | | | | This key context can be used by actions to share partial key computations, for instance when computing MD5s for nested sets. RELNOTES: None PiperOrigin-RevId: 177359607
* Support nested set serialization.Gravatar tomlu2017-11-27
| | | | PiperOrigin-RevId: 177032673
* Remove the deprecated set constructor from SkylarkGravatar vladmos2017-11-20
| | | | | | | | | | | | | The `set` constructor used to be deprecated, but it was still possible to use it by providing --incompatible_disallow_set_constructor=false. It's still allowed to have `set` in parts of the code that are not executed, this will be deprecated later. RELNOTES[INC]: The deprecated `set` constructor is removed, along with the migration flag --incompatible_disallow_set_constructor. It is still temporarily allowed to refer to `set` from within unexecuted code. PiperOrigin-RevId: 176375859
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Automated rollback of commit 1b98de65873054b148ced772cfa827a7bfb5ad9a.Gravatar dslomov2017-10-23
| | | | | | | | | | | | | | | | | | | *** Reason for rollback *** If the 'set' function was used in a .bzl file but not called, --incompatible_disallow_set_constructor=True would allow the load of that .bzl file without error, but this change removes the 'set' function so loading that bzl file is an error. Example failure: https://ci.bazel.io/blue/organizations/jenkins/Global%2FTensorFlow/detail/TensorFlow/245/pipeline/ *** Original change description *** Remove the deprecated set constructor from Skylark The `set` constructor used to be deprecated, but it was still possible to use it by providing --incompatible_disallow_set_constructor=false. RELNOTES[INC]: The flag --incompatible_disallow_set_constructor is no longer available, the deprecated `set` constructor is not available anymore. PiperOrigin-RevId: 173115983
* Remove the deprecated set constructor from SkylarkGravatar vladmos2017-10-13
| | | | | | | | | | The `set` constructor used to be deprecated, but it was still possible to use it by providing --incompatible_disallow_set_constructor=false. RELNOTES[INC]: The flag --incompatible_disallow_set_constructor is no longer available, the deprecated `set` constructor is not available anymore. PiperOrigin-RevId: 171962361
* More BUILD file refactorings.Gravatar philwo2017-09-06
| | | | | | | | | Split collect, concurrent, vfs, windows into package-level BUILD files. Move clock classes out of "util", into their own Java package. Move CompactHashSet into its own Java package to break a dependency cycle. Give nestedset and inmemoryfs their own package-level BUILD files. PiperOrigin-RevId: 167702127
* Fix assorted ErrorProne warnings.Gravatar lberki2017-09-06
| | | | | RELNOTES: None. PiperOrigin-RevId: 167574104
* PolishingGravatar Jonathan Bluett-Duncan2017-09-05
| | | | | | | | This is a follow-on to https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/bazel-dev/Q2owiR-e86s/ugrVUhn7AwAJ to introduce more usages of Java 8 idioms and other "cleanups", with the intention of making the code base easier to maintain. Closes #3623. PiperOrigin-RevId: 167566256
* Deprecate old ordering names for depsetsGravatar vladmos2017-08-10
| | | | | | | | | Old ordering names ("stable", "compile", "naive_link", "link") are deprecated and won't be available if --incompatible_disallow_set_constructor=true is set. Use "default", "postorder", "preorder", and "topological" correspondingly instead. PiperOrigin-RevId: 164728439
* Automated conversion to Java 8Gravatar laurentlb2017-06-30
| | | | | | | With a few manual fixes for readability. RELNOTES: None. PiperOrigin-RevId: 160582556
* Extract ImmutableSharedKeysMap class from TransitiveInfoProviderMap.Gravatar Googler2017-06-20
| | | | PiperOrigin-RevId: 159498323
* Clarify commentGravatar brandjon2017-05-03
| | | | | RELNOTES: None PiperOrigin-RevId: 154877525
* Avoid calls to NestedSetBuilder#addAll(NestedSet)Gravatar cushon2017-04-10
| | | | PiperOrigin-RevId: 152654844
* Add a class providing a view of a NestedSetGravatar Klaus Aehlig2017-04-06
| | | | | | | | | | | | | When serializing a large number of related nested sets needs to be serialized in an efficient way, it is necessary to access the internal structure of a nested set to efficiently deduplicate shared content. Add a new class that provides such a view on a nested set. Note: part of this change (in particular, the addition of the NestedSetView class) was accidentally committed as part of 617bb896dc5d2b815449459e991c577237d7a7fc. Change-Id: I03660a228a66bbd6d3df2d3e78e8349be2d55f41 PiperOrigin-RevId: 152362816
* BEP: provide events a generic converter classGravatar aehlig2017-04-04
| | | | | | | | | | | Change the BuildEvent interface to accept a generic class of converters. In this way, we won't have to change it again in the future, once more converters are needed. In fact, a new converter is needed right now (will be added in a follow-up patch) to allow build events to know the name of named artifact groups already reported in the stream. Change-Id: Ibb32ea5fff361e21bcf2d34818d8351a1da7a2e3 PiperOrigin-RevId: 152131870
* Avoid expanding NestedSets in IterablesChain BuilderGravatar Michajlo Matijkiw2017-03-14
| | | | | | | | Iterables.isEmpty winds up expanding the NestedSet, which isn't cheap. -- PiperOrigin-RevId: 150079225 MOS_MIGRATED_REVID=150079225
* Refactor SkylarkNestedSet to not implement IterableGravatar Jon Brandvein2017-01-20
| | | | | | | | | | | | | | This is not intended to be a user-visible semantic change, aside from error messages. This is to help avoid unintentional flattening of depsets, and to narrow down the number of call sites where this can occur, to help us print warning/deprecation messages. EvalUtils#toIterable will now return an ImmutableList in place of SkylarkNestedSet. This should be ok since the caller shouldn't be relying on the result being a Skylark-safe type. Code that takes Iterable because it accepts either a list or set, can instead be changed to take Object and use EvalUtils#toIterableStrict for validation. Note that NestedSet still implements Iterable, so native code can still easily and accidentally flatten sets. -- PiperOrigin-RevId: 145044023 MOS_MIGRATED_REVID=145044023
* Add new names for depset traversal ordersGravatar Jon Brandvein2017-01-20
| | | | | | | | | | The old names are deprecated but still supported for now. A flag for enforcing the deprecation will come in a later change. RELNOTES[INC]: The traversal orders for depsets have been renamed. The old names are deprecated and will be removed in the future. New names: "stable" -> "default", "compile" -> "postorder", "link" -> "topological", "naive_link" -> "preorder". -- PiperOrigin-RevId: 145004718 MOS_MIGRATED_REVID=145004718
* Refactor NestedSetBuilderGravatar Jon Brandvein2017-01-13
| | | | | | | | Add Order#isCompatible, clean javadoc. Also fix broken @Deprecated-based canary for detecting accidental flattening. -- PiperOrigin-RevId: 144331341 MOS_MIGRATED_REVID=144331341
* Short-circuit visitation of empty nested sets, reinstating the spirit of ↵Gravatar Eric Fellheimer2016-06-27
| | | | | | | commit 7c991c333999f3027d31da6af700f575924066cd. -- MOS_MIGRATED_REVID=125969403
* --Gravatar Carmi Grushko2016-06-24
| | | | MOS_MIGRATED_REVID=125712280
* Restore comments documenting the NestedSet orderings (accidentallyGravatar Googler2016-06-20
| | | | | | | removed along with the per-order expander classes). -- MOS_MIGRATED_REVID=125215096
* Description redacted.Gravatar Googler2016-06-16
| | | | | -- MOS_MIGRATED_REVID=125013752
* Review a number of action subclasses and update them according to the spec.Gravatar Ulf Adams2016-04-19
| | | | | | | | Consists of adding @Immutable annotations, adding final modifiers, and changing the types of fields to immutable types. -- MOS_MIGRATED_REVID=120221067
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-11 15:05:04 +0000