aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/test/java/com/google/devtools/build/lib/skyframe
Commit message (Collapse)AuthorAge
* ToolchainContext directly implements SkylarkValue and SkylarkIndexable.Gravatar John Cater2018-06-22
| | | | | Change-Id: I355b138e143771fd826ab03951df821ea7d58ac5 PiperOrigin-RevId: 201740564
* Move HashFunction out of FileSystem, and turn it into a class, instead of an ↵Gravatar ccalvarin2018-06-21
| | | | | | | | | enum. Now that we aren't using enum names for the hash functions, we also accept the standard names, such as SHA-256. RELNOTES: None. PiperOrigin-RevId: 201624286
* Automated rollback of commit 45b308a62f42c2c0bcfe79dcd4046c4025a31059.Gravatar janakr2018-06-20
| | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks Javascript compilation. There's currently no way to iterate over a depset of Artifacts and deduplicate by identical paths in Skylark. This means that actions that want to do something once per Artifact in a depset (add a flag to the command line with the path of the Artifact for instance) will have duplicate entries if there are multiple Artifacts with the same path. This is not a true automated rollback, since I tried to make this as minimal as possible, to avoid merge conflicts and keep some of the benefits of the rolled back CL. In particular, the tests that were changed in the original CL to give artifacts their correct owners did not need to be changed back, since the owners are still correct. Moreover, this effectively contains rollbacks of unknown commit and https://github.com/bazelbuild/bazel/commit/39d6f89644107a8f7c080c4f4aec8527c1a73954, but keeps test coverage from those CLs as well. Comments added to CL thread where not a clean rollback (there are plenty of files not changed at all: ActionArtifactCycleReporter is the main wart, since it can still handle SkyKeys with Artifact as the argument, but Artifact is no longer the argument to any SkyKey). RELNOTES: None *** Original change description *** Make Artifact#equals take the owner into account for derived artifacts. Derived artifacts' owners are important because they are used to determine the artifact's generating action. Source artifacts' owners are not used in this way, so I left them alone. This allows us to get rid of most uses of ArtifactSkyKey. We may be able to delete it entirely in a follow-up. PiperOrigin-RevId: 201464780
* Add new BuildMetrics event to BEP.Gravatar tomlu2018-06-19
| | | | | | | To start we add just a single metric, the number of actions constructed in the current build. RELNOTES: None PiperOrigin-RevId: 201248490
* Expand the abilities of codecs. Let ImmutableMultimapCodec actually handle ↵Gravatar janakr2018-06-19
| | | | | | | | LinkedHashMultimap as well. Let ImmutableSetRuntimeCodec handle some other sets. We squash these extra types into the same codec because we want them to become immutable anyway, and they can be dealt with easily by the existing codecs. Add some more non-Bazel classes that should be DynamicCodec'ed. Also, when a disallowed value is serialized, throw a SerializationException instead of crashing. This makes it easier to recover in tests. PiperOrigin-RevId: 201237631
* Put a trail into every SerializationException, not just NoCodecException, ↵Gravatar janakr2018-06-19
| | | | | | and have DynamicCodec add to it. It's very useful to have this trail available when debugging or whitelisting. PiperOrigin-RevId: 201205884
* Remove support for --discard_actions_after_execution.Gravatar tomlu2018-06-18
| | | | | | | The memory savings from this flag are not worth the complexity, and it interferes with action restarting. RELNOTES: Remove support for --discard_actions_after_execution. PiperOrigin-RevId: 201077905
* Refactoring: uses OutputService for ActionFileSystem injectionGravatar shahan2018-06-18
| | | | | | Moves more Action-oriented from skyframe package, which has become very unwieldy, to action package. This is needed to avoid circular dependencies caused when build-base is needed for skyframe. PiperOrigin-RevId: 200996982
* ActionFS supports fast digests. Add test coverage for this and some file ↵Gravatar felly2018-06-15
| | | | | | | metadata operations. RELNOTES: None PiperOrigin-RevId: 200765925
* Support ActionFS opening files with append. Also, fix ActionFS notifying its ↵Gravatar felly2018-06-15
| | | | | | | metadata consumer multiple times per inline output. RELNOTES: None PiperOrigin-RevId: 200730252
* Create an ImmutableTableCodec.Gravatar mjhalupka2018-06-14
| | | | PiperOrigin-RevId: 200561008
* Support unconditional fetching of repositoriesGravatar Klaus Aehlig2018-06-14
| | | | | | | | | | | Make all external repositories depend on an additional SkyValue controllable via commands, so support unconditional fetching of all external repositories, as it is needed by the the `sync` command. Improves on #5175, provides a work around for #4907. Change-Id: I30033614c1a2fad3f1363b85ff69cf92f697c255 PiperOrigin-RevId: 200543985
* Serialize lambdas when they are cast to Serializable.Gravatar janakr2018-06-13
| | | | | | We could conceivably do some monkey-patching at server startup to make all lambdas act Serializable, but one step at a time. PiperOrigin-RevId: 200509321
* Add functionality to make certain SkyValues unshareable, meaning they are ↵Gravatar janakr2018-06-13
| | | | | | not serialized. Tag TestCompletionValue and any ActionExecutionValue coming from a NotifyOnActionCacheHit (i.e., tests) like that. To make such values really not shared, request the ActionExecutionValue from TestCompletionFunction as opposed to the ArtifactValue (propagating the unshareable bit up seemed like too much fuss, and I have a dream of getting rid of ArtifactValue anyway). PiperOrigin-RevId: 200504358
* Fix a bug with flush() not flushing on local writes in ActionFS, and add ↵Gravatar felly2018-06-12
| | | | | | | | | unit test coverage for this. Note that ActionFS is not generic enough to make use of FileSystemTest. RELNOTES: None PiperOrigin-RevId: 200304871
* Add more data to the exception thrown when putOnlyEntry's verification fails.Gravatar mstaib2018-06-12
| | | | | | | | | If this exception is thrown, performance no longer matters - we're detonating the place and riding the explosion out to stderr. So we might as well just dump everything we know. RELNOTES: None. PiperOrigin-RevId: 200290439
* Always run toolchain resolution, even when no toolchain types are requested, ↵Gravatar John Cater2018-06-12
| | | | | | | in order to properly choose the execution platform from the available execution platforms. Change-Id: I05dc84403e0db765865e9b91c4222894fa867cd9 PiperOrigin-RevId: 200211635
* Allow delegation to the underlying filesystem only for the source tree.Gravatar felly2018-06-11
| | | | | | | The on-disk execRoot/blaze-out is now off limits to ActionFS. RELNOTES: None PiperOrigin-RevId: 200080287
* Make Artifact#equals take the owner into account for derived artifacts.Gravatar janakr2018-06-08
| | | | | | | | Derived artifacts' owners are important because they are used to determine the artifact's generating action. Source artifacts' owners are not used in this way, so I left them alone. This allows us to get rid of most uses of ArtifactSkyKey. We may be able to delete it entirely in a follow-up. PiperOrigin-RevId: 199836436
* Deletes ActionInputFileCache.Gravatar shahan2018-06-07
| | | | PiperOrigin-RevId: 199732415
* Refactoring: makes the code simpler by deleting Metadata and InputArtifactData.Gravatar shahan2018-06-06
| | | | | | (minor) ActionFS now implements MetadataProvider.getInput PiperOrigin-RevId: 199575194
* Disable `repo_mapping` by default via a flag.Gravatar dannark2018-06-05
| | | | | RELNOTES: None PiperOrigin-RevId: 199374368
* Allow rules and targets to specify extra execution platform constraints.Gravatar John Cater2018-06-05
| | | | | | | | | | | | | | | | | | | | | | RuleClass.Builder now allows authors to specify whether a rule's targets can add additional constraints on the execution platform, and to declare additional constraints for all targets of that rule. Targets which support this now have an attribute, "exec_compatible_with", which supports specifying additional constraints that the execution platform used must match. This attribute is non-configurable. It will only affect execution platforms used during toolchain resolution. Part of #5217. Change-Id: Id2400dbf869a00aa2be3e3d2f085c2850cd6dc00 Closes #5227. Change-Id: If7d55f08f7f44bc7d7f6dfec86a3e6bcd68574b9 PiperOrigin-RevId: 199326255
* Add functionality to SerializationContext and @AutoCodec to check that a ↵Gravatar janakr2018-06-05
| | | | | | | | class is allowed to be serialized in the current context. A codec can now add an explicitly allowed class that can be serialized underneath it (via SerializationContext#addExplicitlyAllowedClass), and that class's codec can check that it is explicitly allowed (via SerializationContext#checkClassExplicitlyAllowed). It is a runtime crash if a codec checks that it was explicitly allowed and finds that it wasn't. Thus, if PackageCodec is invoked without it having been explicitly allowed, we will crash, preventing Package from sneaking into a value it shouldn't be in. This is only enabled if the codec is memoizing. PiperOrigin-RevId: 199317936
* Process 'repo_mapping' attribute from WORKSPACE rules. 'repo_mapping' is a ↵Gravatar dannark2018-06-04
| | | | | | | | | | | | | | | | way to remap references to repositories within an external repository by another name. This CL only adds the mappings to the Package object, but it does not actually enable the reassignments. Example usage (in a WORKSPACE file): local_repository( name = ?a?, path = ?../a?, repo_mapping = {?@x? : ?@y?} ) This change also creates a new SkyKey which represents the mappings. This is to prevent all packages from depending on the external package, and instead depending just on the mappings. i.e. a change to the WORKSPACE file that does not touch the mappings shouldn't cause a reload of the package. RELNOTES:None PiperOrigin-RevId: 199187963
* Remove ConfiguredTarget from TargetCompletionValue, since it is no longer ↵Gravatar janakr2018-06-04
| | | | | | needed: we can get the ConfiguredTargetKey directly from the TargetCompletionKey. Since that was the only use of the actual value in EvaluationProgressReceiver#evaluated, remove it, instead just provide a boolean enum that gives whether or not evaluation succeeded. PiperOrigin-RevId: 199178047
* Failing targets are identified by comparing ConfiguredTarget.Key instead of ↵Gravatar cpeyser2018-06-04
| | | | | | ConfiguredTarget. Since ConfiguredTarget does not implement equals(), reference equality has been used until now, and this is not valid for the case where the ConfiguredTarget has been deserialized. PiperOrigin-RevId: 199133926
* Carry tree artifacts' self data through aggregating middlemen.Gravatar Benjamin Peterson2018-06-03
| | | | | | | | | | | | | | | | | | | | | | | | This CL is a followup to c868c47. It is intended to fix #5296. The immediate problem in that issue is that remote caching finds a tree artifact in an input runfiles tree without metadata in the cache. The metadata isn't there because the runfiles artifacts are now all behind a middleman action, and ArtifactFunction didn't propagate tree artifact values through middlemen. This change fixes the problem by adding a tree artifact's self data value to the values carried through to dependent actions. Remote caching with tree artifact inputs seems to have been working mostly by accident. (Evidence: it wasn't tested, and remote execution of actions with tree artifact inputs results a Java traceback before and after c868c47.) This CL is only really a bandaid to return to the old "working" state. If remote execution wants to start supporting tree artifacts properly in the future, we'll likely need to carry the tree artifact child data through middleman actions, too. Closes #5299. Change-Id: I2e07d4fca0f6d2d34d97b7edb27f9d0063776225 PiperOrigin-RevId: 199079382
* Allow multiple trimming transition factories to be added.Gravatar mstaib2018-06-01
| | | | | | | | If multiple trimming transition factories are added, they are composed in the order they were added. RELNOTES: None. PiperOrigin-RevId: 198934666
* Refactor root cause reporting in ConfiguredTargetFunctionGravatar ulfjack2018-05-29
| | | | | | We now track Causes instead of plain Labels, which will allow us to do better reporting in the future. Add basic tests. PiperOrigin-RevId: 198380468
* Simplify split transition semantics: noops now return the input build options.Gravatar gregce2018-05-23
| | | | | | This results in less special logic in the implementation and a simpler API. PiperOrigin-RevId: 197772283
* Rename PatchTransition.apply to PatchTransiton.patch.Gravatar gregce2018-05-23
| | | | | | | This unclashes with the incoming ConfigurationTransition.apply method described in https://docs.google.com/document/d/1_UJKmAQ9EE8i3Pl0il3YLTYr-Q9EKYYyLatt2zohfyM/edit#heading=h.96gongkwg852. PiperOrigin-RevId: 197769784
* Remove string prefix optimization. With new UnsafeJdk9StringCodec, it's no ↵Gravatar janakr2018-05-21
| | | | | | longer worth it. PiperOrigin-RevId: 197477783
* Remove ValueConstants. They're not pulling their weight in CPU overhead.Gravatar janakr2018-05-21
| | | | PiperOrigin-RevId: 197465288
* When using JDK9, replace naive StringCodec with an optimized codec that uses ↵Gravatar janakr2018-05-21
| | | | | | the String's underlying raw bytes. This avoids byte copying and UTF interpretation. Experiments show it is approximately 15(!) times faster than the naive StringCodec for serialization and 2 times faster for deserialization (10 times faster for non-ASCII strings). PiperOrigin-RevId: 197441758
* Migrate some serialization tests to use SerializationTester rather than ↵Gravatar janakr2018-05-15
| | | | | | | | AbstractObjectCodecTest. I saw these tests causing errors in one version of the sequel unknown commit when testing detection of junk data, but not with this clean-up. Also use the String codec provided in the context in AutoCodecProcessorTest. PiperOrigin-RevId: 196756537
* Optimizes performance of ActionFS staging and eliminates ActionFS updates.Gravatar shahan2018-05-15
| | | | | | | | | Extracts a class, InputArtifactData to hold the input data instead of using a raw map. This provides the flexibility needed to support both ActionFS and existing code so ActionFS does not need to rekey the input data. Uses the smaller, getDeclaredIncludeSrcs instead of getAllowedDerivedInputs when possible for staging optional inputs in ActionFS. PiperOrigin-RevId: 196736703
* Allow more characters in labels.Gravatar twerth2018-05-15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Partly addresses #374. Specifically allow !%^`"'&;<>?[]{|} in target and package names. It's actually simpler now to declare what we don't allow. In target names: 0-31 (control characters) 58 ':' (colon) 92 '\' (backslash) 127 (delete) In package names: 0-31 (control characters) 58 ':' (colon) 64 '@' (at-sign) 92 '\' (backslash) 127 (delete) - '\' is a path segment separator on Windows, and allowing it can lead to silent output file conflicts and - therefore - data corruption. We may be able to allow it in the future, but I didn't want to make that call. - ':' is a special character that Bazel interprets as the package name / target name separator. - '@' in package names can probably be allowed; at the beginning of a label it indicates a workspace name, but not within a segment. We actually have some tests that disallow it specifically, but those can probably just be deleted; however, it does require a bit of investigation, so I decided to delay that change. It is possible that we don't correctly escape filenames in all cases. Also note that the shell may require escaping for specific characters, and that Bazel treats a single '*' (star) target name specially when given on the command line. RELNOTES: Bazel now allows almost all 7-bit ASCII characters in labels. PiperOrigin-RevId: 196650651
* Use the local JDK as the default target javabaseGravatar cushon2018-05-14
| | | | | | and continue to use the embedded JDK as the default host_javabase. PiperOrigin-RevId: 196471714
* Codec for FluentIterable - needed to serialize results of Iterables.concat.Gravatar shahan2018-05-10
| | | | PiperOrigin-RevId: 196127040
* Adds an action-scoped filesystem.Gravatar shahan2018-05-09
| | | | PiperOrigin-RevId: 195973862
* Use the in-memory metadata in blaze as the source of truth for Fileset mappingsGravatar kush2018-05-02
| | | | | | | instead of the manifest files. RELNOTES: None PiperOrigin-RevId: 195149880
* Add a codec for LinkedHashSet.Gravatar mjhalupka2018-05-01
| | | | PiperOrigin-RevId: 195002908
* Reduce sensitivity of certain tests to exact configurations.Gravatar mstaib2018-04-27
| | | | | | | | | | Because trimming will impact these tests, rephrasing them so that they don't depend on the exact configuration (but instead the actual properties they're trying to test) allows them to pass when the trimming transition is in place. RELNOTES: None. PiperOrigin-RevId: 194552473
* Allow class prefix names to be blacklisted from DynamicCodec use.Gravatar janakr2018-04-26
| | | | PiperOrigin-RevId: 194487570
* Replace ConfigurationTransition with PatchTransition in a few places.Gravatar mstaib2018-04-26
| | | | | RELNOTES: None. PiperOrigin-RevId: 194459347
* RuleTransitionFactory returns NoTransition instead of null.Gravatar mstaib2018-04-26
| | | | | RELNOTES: None. PiperOrigin-RevId: 194433721
* Add the capability to register a trimming RuleTransitionFactory.Gravatar mstaib2018-04-19
| | | | | | | | | | This RuleTransitionFactory will be applied to all targets after other transitions, and is intended to be used to manually trim the configuration based on tagging of that target. This is a stopgap feature until automatic trimming of configuration can be implemented. RELNOTES: None. PiperOrigin-RevId: 193573013
* Introduce PackageErrorMessageValueGravatar mschaller2018-04-19
| | | | | | | | | | This CL introduces a new intermediate SkyValue type, PackageErrorMessageValue, to be used by RecursivePkgFunction. RecursivePkgValue will now have a direct dep on PackageErrorMessageValue rather than PackageValue. RELNOTES: None PiperOrigin-RevId: 193549158
* Automated rollback of commit 559ffb7dd6578c961e775ba4901917ab8bffcb9d.Gravatar janakr2018-04-18
| | | | | | | | | | | | *** Reason for rollback *** Looks like this is using more cpu than I realized. Doesn't seem fixable short-term. This may need to be cherrypicked into the next canary if it shows up in profiling. See bug for details. *** Original change description *** Add ValueConstants helper to handle the increasingly complex logic for value-equality-tested constants. Main hurdle is efficiently testing to see if a Collection is a value constant without trying to do work on Collections that can't possibly be value constants. PiperOrigin-RevId: 193403226
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-08 00:16:29 +0000