aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/test/java/com/google/devtools/build/lib/skyframe
Commit message (Collapse)AuthorAge
* Automated rollback of commit 559ffb7dd6578c961e775ba4901917ab8bffcb9d.Gravatar janakr2018-04-18
| | | | | | | | | | | | *** Reason for rollback *** Looks like this is using more cpu than I realized. Doesn't seem fixable short-term. This may need to be cherrypicked into the next canary if it shows up in profiling. See bug for details. *** Original change description *** Add ValueConstants helper to handle the increasingly complex logic for value-equality-tested constants. Main hurdle is efficiently testing to see if a Collection is a value constant without trying to do work on Collections that can't possibly be value constants. PiperOrigin-RevId: 193403226
* Add ValueConstants helper to handle the increasingly complex logic for ↵Gravatar janakr2018-04-16
| | | | | | value-equality-tested constants. Main hurdle is efficiently testing to see if a Collection is a value constant without trying to do work on Collections that can't possibly be value constants. PiperOrigin-RevId: 193085625
* bash: Add a toolchain for local Bash.Gravatar Laszlo Csomor2018-04-16
| | | | | | | | | | | | | | | | | | | Bazel automatically detects the local Bash and creates a custom toolchain rule for it. Later, rules that use Bash will require this toolchain and retrieve Bash's path from it instead of relying on hardcoded paths or the `--shell_executable` flag. See https://github.com/bazelbuild/bazel/issues/4319 Change-Id: Idd8242a20d202b1f5a56cddac95b625c6c08ede9 Closes #4980. Change-Id: Ic2406a4da260b284e15852070d58472ca18340af PiperOrigin-RevId: 193022708
* Add DurationCodecGravatar Googler2018-04-13
| | | | | | This is required for moving --test_timeout from ExecutionOptions to BuildConfiguration.Options PiperOrigin-RevId: 192815911
* Prefix-compress strings during memoizing serialization.Gravatar janakr2018-04-12
| | | | PiperOrigin-RevId: 192662977
* Add value constants to ObjectCodecRegistry. Value constants are to be used ↵Gravatar janakr2018-04-10
| | | | | | when there may not be a canonical instance of the object we want (or the canonical instance isn't available at registry construction time), but we can reasonably cheaply do value equality comparisons. Strings are a good example. PiperOrigin-RevId: 192354865
* Make some objects frequently encountered during serialization into ↵Gravatar janakr2018-04-10
| | | | | | | | constants. A lot of care is needed here because we're using reference equality. I plan to add value-equality constants in a follow-up. Add ImmutableSortedSet marshaller because I think it might have been needed, and hey, why not. PiperOrigin-RevId: 192326359
* DynamicCodec emits a trail of type names when it encounters ↵Gravatar shahan2018-04-06
| | | | | | | | NoSuchCodecException. This will make it easier to trace down missing codecs. PiperOrigin-RevId: 191920743
* Make DynamicCodec always memoize. Since it replaces Java serialization, ↵Gravatar janakr2018-04-05
| | | | | | which memoizes, we should too. PiperOrigin-RevId: 191813677
* Transform StackOverflowError thrown in ArrayCodec into ↵Gravatar janakr2018-04-05
| | | | | | SerializationException. This can happen with deep nested sets. PiperOrigin-RevId: 191790431
* Add codec for Object[]. This may be useful for NestedSetCodec, pushing up so ↵Gravatar janakr2018-04-04
| | | | | | | | that my uncommitted change chain is a little more manageable. Unfortunately we can't get rid of the array marshaller because this only handles Object[], not any other type of array. PiperOrigin-RevId: 191662535
* Save the lookup and write of the codec index when the object is memoized, as ↵Gravatar janakr2018-04-04
| | | | | | well as the NEW_VALUE/BACKREF enum write. Probably doesn't actually take much space/time, but free optimization. PiperOrigin-RevId: 191633484
* Add ObjectCodec for AtomicReference.Gravatar janakr2018-04-04
| | | | PiperOrigin-RevId: 191615301
* Replaces JavaSerializableCodec with DynamicCodec as the defaultGravatar shahan2018-04-04
| | | | | | | | | * Skylark serialization was previously dropping location in error, which this fixes. * Deletes a lot of codecs with fidelity issues (DynamicCodec has full fidelity). * Deletes EnumRuntimeCodec which can now be replaced with the superior EnumCodec. * This should eventually allow us to delete Serializable from all Blaze. The remaining blocker is NoSuchPackageExceptionCodec. PiperOrigin-RevId: 191603929
* BazelPackageLoader to be invoked on additional tests.Gravatar carmi2018-04-03
| | | | | | | I originally blacklisted BazelPackageLoader on some tests because they became flaky with it. RELNOTES: None PiperOrigin-RevId: 191504852
* Fix build results for aspect builds.Gravatar tomlu2018-04-03
| | | | | | | | | The current output was pretty much completely incorrect. However since the result output was always hidden for the default value of --show_result, users simply didn't see the incorrect output (instead getting no output at all). This CL fixes both the --show_result problem and makes the output correct. RELNOTES: Print correct build result for builds with --aspects flag. PiperOrigin-RevId: 191456352
* Remove some unnecessary params.Gravatar nharmata2018-04-02
| | | | | RELNOTES: None PiperOrigin-RevId: 191354727
* Moves the decision to enable memoization from codecs to the top-level ↵Gravatar shahan2018-04-02
| | | | | | | | invocation. Also, makes it benign to registerInitialValue when memoization is disabled. PiperOrigin-RevId: 191338253
* Add codec for ImmutableMultimap.Gravatar cpeyser2018-04-02
| | | | PiperOrigin-RevId: 191323243
* Don't test for reference equality of the underlying IOException. It's ↵Gravatar nharmata2018-03-30
| | | | | | | overkill for these unit tests. RELNOTES: None PiperOrigin-RevId: 191122418
* Make RuleClass serializable and remove Environment from it, since it was ↵Gravatar janakr2018-03-30
| | | | | | | | only being used for the transitive hash code and transitive label of its globals, which can be passed in explicitly. Assert along the way that the transitive label of its globals is always non-null. That is currently the case, although there seems to be no hard invariant of the system that it is true. Might as well tighten it now. PiperOrigin-RevId: 191103310
* Add codecs needed to fully serialize Attributes.Gravatar janakr2018-03-28
| | | | PiperOrigin-RevId: 190805577
* Removes superfluous generic parameter from DynamicCodec, allowing it to be usedGravatar shahan2018-03-28
| | | | | | with CodecRegisterer. PiperOrigin-RevId: 190780559
* ArtifactRoot correctly switches output base when serializing.Gravatar shahan2018-03-28
| | | | PiperOrigin-RevId: 190779535
* Replace nearly all test uses of ConfiguredTarget#getConfiguration with ↵Gravatar janakr2018-03-27
| | | | | | | | | | utility method getConfiguration(ConfiguredTarget). Done using IntelliJ structural replace, with $val$.getConfiguration() -> getConfiguration($val$) and Expression type: TransitiveInfoCollection, applying constraint within type hierarchy. Had to clean up a few compile errors. PiperOrigin-RevId: 190692493
* DynamicCodec class.Gravatar shahan2018-03-27
| | | | PiperOrigin-RevId: 190667019
* New incompatible flag to disable PACKAGE_NAME and REPOSITORY_NAME.Gravatar laurentlb2018-03-27
| | | | | | | | | RELNOTES[INC]: Variables PACKAGE_NAME and REPOSITORY_NAME are deprecated in favor of functions `package_name()` and `repository_name()`. https://docs.bazel.build/versions/master/skylark/lib/native.html#package_name PiperOrigin-RevId: 190657188
* Add toolchain definitions for autoconfigured cc_toolchain entries.Gravatar jcater2018-03-27
| | | | PiperOrigin-RevId: 190611069
* Remove FastStringCodecGravatar michajlo2018-03-26
| | | | | | Currently unnecessary PiperOrigin-RevId: 190568226
* @AutoCodec SelectorValue. Rather than making mapMarshaller handle wildcards ↵Gravatar janakr2018-03-23
| | | | | | better, get rid of it, and immutableSetMarshaller for good measure. PiperOrigin-RevId: 190294922
* Fix test flakiness due to the filesystem mutations done during the 'fetch' ↵Gravatar nharmata2018-03-23
| | | | | | | | | work done by BazelPackageBuilderHelperForTesting#sanityCheckBazelPackageLoader. We fix this flakiness by _not_ having RepositoryDelegatorFunction unconditionally 'fetch' local repos. We then have to make a change to BazelPackageLoader test to reinstate these mutations, since it was only working before due to the unconditional filesystem mutations done here :p RELNOTES: None PiperOrigin-RevId: 190222830
* Remove RuleClass.Builder#depsCfg().Gravatar lberki2018-03-23
| | | | | | | The only user of it was Android resource trimming, which just got deleted. RELNOTES: None. PiperOrigin-RevId: 190193627
* Add a DefaultBuildOptions specifying default build flags that may differ ↵Gravatar mjhalupka2018-03-22
| | | | | | from the provided defaults in Options classes. These are used to create BuildOptionsDiffForReconstruction, which lets us store only the diffs in our BuildConfigurationValue.Keys. PiperOrigin-RevId: 190117455
* Clean up unnecessary "additional data" from memoizing deserialization. Since ↵Gravatar janakr2018-03-21
| | | | | | memoization is now a simple on-off switch, change semantics to have at most one memoizing frame: starting memoization is now an idempotent operation. PiperOrigin-RevId: 189993914
* Add EnumMapCodec. Some hackery necessary to get Class for Enum when map is ↵Gravatar janakr2018-03-20
| | | | | | | | | | empty. Since EnumMap.equals() ignores the key type when the maps are empty [1], we could just return a canonical empty EnumMap, but that seems to depend on implementation details in a hacky and potentially subtly broken way. The Unsafe method will hopefully break spectacularly if it goes wrong. [1] http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/util/EnumMap.java#l685 PiperOrigin-RevId: 189803280
* Makes leaf level AutoCodec marshallers into runtime codecs.Gravatar shahan2018-03-15
| | | | | | Next change will convert containers. PiperOrigin-RevId: 189261293
* Allow BazelPackageLoader to load external repositories.Gravatar carmi2018-03-13
| | | | | | | | | Also, disallow BazelPackageLoader from fetching missing external repos. Integration tests for BazelPackageLoader wrt external repos will be left for a follow-up CL. RELNOTES: None. PiperOrigin-RevId: 188967694
* Integrate memoization into standard serialization. This involves a number of ↵Gravatar janakr2018-03-13
| | | | | | | | | | | | | | | | large changes: 1. SerializationContext and DeserializationContext become the owners of the Memoizer if requested. They produce new versions of themselves on demand that are memoization-aware. Because of intricacies of Skylark that I do not fully understand, we inject a Mutability object when starting memoization, and so to be conservative, that injection starts up a new memoization frame, nested if we were already memoizing, just like before. It would be nice to decouple this injection from memoization in the future. 2. MemoizingCodec is deleted, but really ObjectCodec becomes MemoizingCodec, so it lives on. BaseCodec is deleted since it now has only one implementation. 3. The simplified model of registering MemoizingCodecs is adopted for ObjectCodecs: all codecs are registered based on their #getEncodedClass and #additionalEncodedSubclasses. This also allows us to register codecs that are defined using tricky parameter types, since we're no longer trying to reflectively examine them. This required a clean-up of such codecs, and the addition of ArrayListCodec to stop NullableListCodec from making lists unmodifiable when they shouldn't be. 4. @AutoCodec is extended to allow users to specify that memoization should start with this codec. To ensure bit-equivalence, SkyKeySerializer disables memoization. PiperOrigin-RevId: 188918251
* tests: add TimestampGranularityUtilsGravatar laszlocsomor2018-03-13
| | | | | | | | | | | | | | Add a new test utility class TimestampGranularityUtils which offers methods to wait for the filesystem timestamp granularity to pass. Such waiting is necessary if a test asserts changes in a file's mtime or ctime. This is a follow-up to comments on https://github.com/bazelbuild/bazel/commit/a31e035fb42aa8db9fd248ef2ebd665a411053b8. PiperOrigin-RevId: 188847729
* Fix toolchain and execution platform registration to use patterns.Gravatar John Cater2018-03-12
| | | | | | | | This allows more flexibility in registering toolchains and execution platforms, both in the WORKSPACE and from the command-line. Change-Id: I6fe75507d1a74de74085b7c927fdf093c152b894 PiperOrigin-RevId: 188813688
* Get rid of non-class-based lookups for ObjectCodecs. There were basically no ↵Gravatar janakr2018-03-10
| | | | | | | | remaining non-test users (ErrorInfoEncoder was it). Should help to simplify our codec registration. I want to converge MemoizingCodec and ObjectCodec, and the unnecessary complexity around ObjectCodec registration is annoying. PiperOrigin-RevId: 188620988
* Open-source ImmutableMapCodec and make it able to handle arbitrary ↵Gravatar janakr2018-03-10
| | | | | | | | | | keys/values without injecting the codecs. Also allow it to handle ImmutableSortedMap, since we were always silently degrading to ImmutableMap for objects that weren't declared as ImmutableSortedMap, and there's no good way to handle non-natural comparators. This will lead to runtime failures if a class actually needs an ImmutableSortedMap with a different comparator than the natural one. This changes the semantics of ImmutableMap serialization. Previously, we went off the declared type. In the case of a declared ImmutableSortedMap, we ignored the comparator, potentially leading to incorrect serialization (new test added in AutoCodecProcessorTest that would have caught that). Moreover, declared ImmutableMaps were deserialized as ImmutableMaps even if they were actually ImmutableSortedMaps. Now, we preserve the ordering unconditionally, and preserve the type if possible. I think this is a better state to be in. This is needed to kill off MemoizingCodec, since MemoizingCodec has an ImmutableMapMemoizingCodec, which I want to get rid of in favor of this. PiperOrigin-RevId: 188619637
* Add BuildConfiguration to ConfiguredTargetAndTarget and rename it to ↵Gravatar janakr2018-03-10
| | | | | | ConfiguredTargetAndData. We want to get BuildConfiguration out of ConfiguredTarget because it uses >800K when serialized. PiperOrigin-RevId: 188600002
* Allow ObjectCodecRegistry to handle MemoizingCodecs. Initially this is just ↵Gravatar janakr2018-03-09
| | | | | | | | | | | | | | | | Skylark memoization, but we may extend in future to handle more than just Skylark this way. In fact, we probably want most of our ObjectCodecs to be MemoizingCodecs that can efficiently fall back to ObjectCodec if not using memoization (maybe coming in a follow-up). At a high level, this CL merges the functionality of MemoizingCodecMap into ObjectCodecRegistry, adds auto-registration of MemoizingCodec, and uses that to get rid of a lot of codecs that were just delegating. The big one to get rid of there is SkylarkValueCodec: all of its delegation duties are implicitly now in ObjectCodecRegistry and friends. One danger with this CL is that many of the features of Skylark serialization are only being tested in unit tests, which had to be reworked as part of this change. I don't think we've lost any coverage, but I could be wrong. SkylarkValueCodec had a bunch of methods that were effectively test-only, which made it easier to remove. The plan is to provide a Memoizer.Serializer inside the SerializationContext. At the top level, it will be a DUMMY_SERIALIZER that does no memoization, but a MemoizingCodec can do context = context.ensureMemoizing() which will recreate the context with a true memoizing serializer. Then all references to the Serializer in codec code can be cleaned up, and the MemoizingCodec and ObjectCodec signatures will be the same. At that point, we can make all ObjectCodecs compatible with memoization by default (with strategy MEMOIZE_AFTER), and add a "memoize" boolean to @AutoCodec. That should allow us to have full interoperability between all codecs. This CL also makes CodecScanner deterministic in the order of classes that it processes (there was a lurking bug here where constants must be deterministically ordered but that wasn't enforced at all). PiperOrigin-RevId: 188559983
* tests: fix flaky RecursiveFSTraversalFunctionTestGravatar Laszlo Csomor2018-03-09
| | | | | | | | | | | | | | | | | | Fix the testTraversalOfGeneratedDirectory method in RecursiveFilesystemTraversalFunctionTest that was flaky on Windows. The fix is to wait so that changes to files in the InMemoryFileSystem will have observable effects on the file ctimes. Depends on https://github.com/bazelbuild/bazel/pull/4787 Fixes https://github.com/bazelbuild/bazel/issues/4755 Closes #4788. PiperOrigin-RevId: 188470080
* tests: fix flaky FilesystemValueCheckerTestGravatar Laszlo Csomor2018-03-09
| | | | | | | | | | | | | Fix FilesystemValueCheckerTest.testExplicitFiles() by ensuring that the filesystem timestamp granularity has elapsed before attempting to update the files. Fixes https://github.com/bazelbuild/bazel/issues/4755 Closes #4786. PiperOrigin-RevId: 188467381
* tests: fix flaky SkyframeAwareActionTestGravatar Laszlo Csomor2018-03-09
| | | | | | | | | | | | | Fix testCacheBypassingActionWithMtimeChangingInput in SkyframeAwareActionTest by ensuring that enough time elapses between file updates so their effects are observable on the file's ctime. Fixes https://github.com/bazelbuild/bazel/issues/4755 Closes #4787. PiperOrigin-RevId: 188458542
* Expose an actions provider on RuleConfiguredTarget instances.Gravatar cparsons2018-03-06
| | | | | | | | | Given a target (for example from a skylark aspect), one will be able to access a list of actions that the target generated using "target.actions". This is without additional memory footprint. Actions themselves are not fully exposed to skylark (and thus there isn't much meaning to gather from them in skylark yet). Access methods will follow soon. RELNOTES: None. PiperOrigin-RevId: 188098079
* @AutoCodec File{State}Value.Gravatar janakr2018-03-06
| | | | PiperOrigin-RevId: 188085895
* @AutoCodec RepositoryName.Gravatar janakr2018-03-05
| | | | PiperOrigin-RevId: 187956593
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-17 06:01:35 +0000