aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/skyframe/SkylarkImportLookupFunction.java
Commit message (Collapse)AuthorAge
* Refactoring: makes the code simpler by deleting Metadata and InputArtifactData.Gravatar shahan2018-06-06
| | | | | | (minor) ActionFS now implements MetadataProvider.getInput PiperOrigin-RevId: 199575194
* Clean up code that directly imports nested classes like Builder, Entry, etc.Gravatar jcater2018-05-01
| | | | PiperOrigin-RevId: 194998981
* In inlined skylark import lookup function caching, don't actually fetch deps ↵Gravatar shreyax2018-04-06
| | | | | | when all we want to do is register an edge and don't require the value. PiperOrigin-RevId: 191966203
* Automated rollback of commit 72d28f3efc2842510a34cacd930c0204143f7412.Gravatar shreyax2018-03-19
| | | | | | | | | | | | | | | | | | | | | | | | Fix skylark caching to properly include transitive dependencies when there is a diamond-like dependency in the loaded bzl files. Also add guards to make sure we're not attempting to cache skylark files that transitively request a dependency that is in error. *** Reason for rollback *** Looking for source of non-determinism *** Original change description *** Automated rollback of commit 7ba939dfd5df48903929e9c14ebd0449656403e4. *** Reason for rollback *** Likely cause for non-determinism in skyframe *** Original change description *** Cache SkylarkLookupImportValues in memory so that we don't recompute them multiple times. PiperOrigin-RevId: 189686604
* Automated rollback of commit 7ba939dfd5df48903929e9c14ebd0449656403e4.Gravatar shreyax2018-03-12
| | | | | | | | | | | | *** Reason for rollback *** Likely cause for non-determinism in skyframe *** Original change description *** Cache SkylarkLookupImportValues in memory so that we don't recompute them multiple times. PiperOrigin-RevId: 188729929
* Cache SkylarkLookupImportValues in memory so that we don't recompute them ↵Gravatar shreyax2018-03-05
| | | | | | multiple times. PiperOrigin-RevId: 187941859
* Remove testing flag --internal_do_not_export_builtinsGravatar brandjon2017-12-21
| | | | | | | Also add a @VisibleForTesting accessor to SkyframeExecutor. RELNOTES: None PiperOrigin-RevId: 179875655
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Pass SkylarkSemantics through Skyframe instead of the options classGravatar brandjon2017-10-06
| | | | | | | Also remove the use of the @UsesOnlyCoreTypes annotation on SkylarkSemanticsOptions. It was only there to help mark that the options class was safe to put in Skyframe. RELNOTES: None PiperOrigin-RevId: 171248504
* Change Identifier#boundNames to #boundIdentifiers.Gravatar fzaiser2017-08-28
| | | | | RELNOTES: None PiperOrigin-RevId: 166704851
* Add --internal_do_not_export_builtins, for testingGravatar brandjon2017-07-25
| | | | | | | | | This is a temporary flag that causes Extension objects to elide some built-in symbols, for the benefit of tests. Currently it only applies to the "native" builtin for Extensions created by SkylarkImportLookupFunction. A future Skylark language change will have the effect of excluding all builtin symbols from all Extension objects anyway, so this flag will eventually be subsumed by that incompatible change. RELNOTES: None PiperOrigin-RevId: 163016504
* A bunch of unrelated cleanups:Gravatar nharmata2017-06-16
| | | | | | | | | | -Have SkylarkImportLookupFunction include causes in the SkyFunctionExceptions it throws. -Better transitive skyframe error declarations in ASTFileLookupFunction. -Have ErrorInfo differentiate between direct and transitive transience. -Introduce ErrorInfoManager and have ParallelEvaluator/ParallelEvaluatorContext use it. RELNOTES: None PiperOrigin-RevId: 159163186
* Propagate postable events further upGravatar Klaus Aehlig2017-06-14
| | | | | | | | | | | | With the introduction of the ExtendedEventHandler, SkyFunctions were given the possibility to post additional Postable events in addition to the standard events. As SkyFunctions have to be restartable, events are collected first and only posted after the function is finished. Make sure that this also applies to postable events and they are not dropped. Change-Id: Ie1c3a0134935c75ea984fa2cc924e7327a9da81f PiperOrigin-RevId: 158964337
* Make Skylark interpreter read Skylark command-line flagsGravatar brandjon2017-04-30
| | | | | | | | | | | This is the second of two CLs for making command line options able to affect the Skylark interpreter. For the main kinds of evaluation contexts -- package loading, .bzl loading, rule analysis, aspect analysis, and computed defaults -- the SkylarkSemanticsOptions object is retrieved from Skyframe and passed along to the Environment builder. For other contexts such as tests, default values of builtin functions, and standalone Skylark, flags are currently not processed. In the future, we may want to split into separate files the options that affect "pure" Skylark vs the options that affect Bazel-flavored Skylark. One possibility is to subclass SkylarkSemanticsOptions into SkylarkBazelSemanticsOptions, and go through an indirection in SkylarkUtils. We could also pass SkylarkSemanticsOptions to the parser, to support --incompatible_* changes that alter Skylark's syntax. I don't think that's needed at the moment. RELNOTES: None PiperOrigin-RevId: 154628391
* Rollback of commit 34ffc4cab1b6cdde54e5945427b79c24fff39aa5.Gravatar Dmitry Lomov2017-03-02
| | | | | | | | | | | | | | *** Reason for rollback *** Roll-forward of commit 01120026dc313ee7ad9ea95069a29252eb19173b with fix. *** Original change description *** Automated [] rollback of commit 01120026dc313ee7ad9ea95069a29252eb19173b. -- PiperOrigin-RevId: 148897534 MOS_MIGRATED_REVID=148897534
* Rollback of commit 01120026dc313ee7ad9ea95069a29252eb19173b.Gravatar Jon Brandvein2017-03-01
| | | | | | -- PiperOrigin-RevId: 148888469 MOS_MIGRATED_REVID=148888469
* Export exportable values as we go instead of at the end of evaluation.Gravatar Dmitry Lomov2017-03-01
| | | | | | | | | This simplifies implementation of attr.* functions and helps make declared provider implementation simpler. -- PiperOrigin-RevId: 148867326 MOS_MIGRATED_REVID=148867326
* When requesting all transitive traversal values recursively, request ↵Gravatar Janak Ramakrishnan2016-10-12
| | | | | | | | | | | packages/targets in parallel with subdirectories. The previous implementation was vulnerable to incomplete traversal in case of cycles: if a subdirectory had a cycle, the targets in the package would never be requested. Requesting the package first would create a different problem, where the subdirectories would never be requested if the package depended on a cycle. Also stop uniquifying Skylark import cycles when inlining. While uniquification is nice, this was leading to us emitting an error during queries that we weren't actually emitting when not inlining. Putting the cycle into the exception error message should be enough information, and since we emit an error per exception anyway, the number of events will still be lower (although each event might be more verbose). -- MOS_MIGRATED_REVID=135846847
* When inlining imports, avoid visiting Skylark files multiple times that are ↵Gravatar Janak Ramakrishnan2016-10-11
| | | | | | | loaded by a single package. This also fixes a bug in which Skylark code expecting reference equality for an object imported multiple ways were not getting it. -- MOS_MIGRATED_REVID=135738789
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Make labels in .bzl files in remote repos resolve relative to their repoGravatar Kristina Chodorow2016-03-21
| | | | | | | | | | | | | | | | | | For example, if you have a BUILD file that does: load('@foo//bar:baz.bzl', 'my_rule') my_rule(...) If baz.bzl uses Label('//whatever'), this change makes //whatever resolve to @foo//whatever. Previous to this change, it would be resolved to the repository the BUILD file using my_rule was in. RELNOTES[INC]: Labels in .bzl files in remote repositories will be resolved relative to their repository (instead of the repository the Skylark rule is used in). -- MOS_MIGRATED_REVID=117720181
* Don't keep packages in the default repository around after loading.Gravatar Brian Silverman2016-03-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, this would get thrown when referring to the same package from both the main and default repositories: java.lang.IllegalArgumentException: Multiple entries with same key: tools/cpp=/home/brian/971-Robot-Code and tools/cpp=/home/brian/971-Robot-Code at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:136) at com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:98) at com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:84) at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:295) at com.google.devtools.build.lib.buildtool.BuildTool.transformPackageRoots(BuildTool.java:301) at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:209) at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:334) at com.google.devtools.build.lib.runtime.commands.TestCommand.doTest(TestCommand.java:119) at com.google.devtools.build.lib.runtime.commands.TestCommand.exec(TestCommand.java:104) at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:371) at com.google.devtools.build.lib.runtime.BlazeRuntime$3.exec(BlazeRuntime.java:1016) at com.google.devtools.build.lib.server.RPCService.executeRequest(RPCService.java:65) at com.google.devtools.build.lib.server.RPCServer.executeRequest(RPCServer.java:434) at com.google.devtools.build.lib.server.RPCServer.serve(RPCServer.java:229) at com.google.devtools.build.lib.runtime.BlazeRuntime.serverMain(BlazeRuntime.java:975) at com.google.devtools.build.lib.runtime.BlazeRuntime.main(BlazeRuntime.java:772) at com.google.devtools.build.lib.bazel.BazelMain.main(BazelMain.java:55) And this would get thrown for any packages in the main repository loaded from other repositories: java.lang.RuntimeException: Unrecoverable error while evaluating node 'PACKAGE:@//tools/build_rules/go/toolchain' (requested by nodes ) at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:982) at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:499) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IllegalArgumentException: Invalid BUILD file name for package '@//tools/build_rules/go/toolchain': /home/brian/bazel/tools/build_rules/go/toolchain/BUILD at com.google.devtools.build.lib.packages.Package.finishInit(Package.java:299) at com.google.devtools.build.lib.packages.Package$Builder.finishBuild(Package.java:1308) at com.google.devtools.build.lib.skyframe.PackageFunction.compute(PackageFunction.java:501) at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:933) ... 4 more Sponsor's comment: note the abundance of new Label.resolveRepositoryRelative() calls. They are ugly, but it's only making existing ugliness explicit. Yes, we should fix it, especially in the implementation of configurable attributes. Refs #940 -- Change-Id: I8bd7f7b00bec58a7157507595421bc50c81b404c Reviewed-on: https://bazel-review.googlesource.com/#/c/2591 MOS_MIGRATED_REVID=117429733
* Allow load of external label in the WORKSPACE file.Gravatar Damien Martin-Guillerez2016-02-10
| | | | | | | | | | | | | | | | | Issue #824 Step 5. The only thing left to resolve issue #824 are correct handling of the following cases: - Forbidding overloading of a repository outside of the first part of the workspace file (or it will leads to incorrectness issue and the only use-case is the redefinition of the bazel_tools repository). - Better error reporting when trying to refer to a non-existent repository from the workspace file, for now it will print a SkyFrame cycle exception. -- MOS_MIGRATED_REVID=114316464
* Disallow loads from external repos in WORKSPACE files. Currently, attempting ↵Gravatar John Field2016-01-18
| | | | | | | | | to do such a load will result in a skyframe circular dependency exception. As a side effect of this change, SkylarkImportFailedExceptions thrown by SkylarkImportLookupFunction are now caught by PackageFunction and wrapped and rethrown as a PackageFunctionException. Previously, the first exception wasn't caught, generating an uncaught exception error at top level. -- MOS_MIGRATED_REVID=112328755
* Enable label-based Skylark loading. In particular, such labels may reference ↵Gravatar John Field2015-12-22
| | | | | | | | | | | | | | | | | | | files in external repositories. In addition: - Cleaned up and refactored some tests to reflect the new loading behavior. Deferred to future CLs: - Updating Bazel Skylark documentation to reflect the new load form. - Enabling command-line loading of Aspects via labels. RELNOTES: Skylark load statements may now reference .bzl files via build labels, in addition to paths. In particular, such labels can be used to reference Skylark files in external repositories; e.g., load("@my_external_repo//some_pkg:some_file.bzl", ...). Path-based loads are now deprecated and may be disabled in the future. Caveats: Skylark files currently do not respect package visibility; i.e., all Skylark files are effectively public. Also, loads may not reference the special //external package. -- MOS_MIGRATED_REVID=110786452
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Enable load() statement in the WORKSPACE file.Gravatar Damien Martin-Guillerez2015-11-30
| | | | | | | | | | | RELNOTES[NEW]: Skylark macros are now enabled in WORKSPACE file. Design document at https://docs.google.com/document/d/1jKbNXOVp2T1zJD_iRnVr8k5D0xZKgO8blMVDlXOksJg/preview Fixes #337 -- MOS_MIGRATED_REVID=108860301
* Refactor Skylark rules and attributes in preparation to Skylark aspects.Gravatar Dmitry Lomov2015-11-26
| | | | | | | | | | | 1. attr.<type> functions return a wrapper object instead of Attribute.Builder dierctly. 2. RuleClass is created once per the life-time of RuleFunction, during export 3. Attributes are added to the RuleClass at exporting. -- MOS_MIGRATED_REVID=108774581
* Skylark path-based relative loads now work correctly (with the new ↵Gravatar John Field2015-11-18
| | | | | | | | | label-based loading machinery) when the containing file is in a subdirectory of a package. Also, while we're in the neighborhood: correct two nearby tests of relative loads that were passing for the wrong reasons. -- MOS_MIGRATED_REVID=108072189
* Exit with an error message, rather than crashing, when an attempt is made to ↵Gravatar John Field2015-11-16
| | | | | | | load a Skylark file not contained in a package. -- MOS_MIGRATED_REVID=107811126
* Fix crash when two Skylark loads reference the same path.Gravatar John Field2015-11-16
| | | | | -- MOS_MIGRATED_REVID=107808413
* Use Labels, rather than PathFragments, to represent Skylark loads ↵Gravatar John Field2015-11-13
| | | | | | | | | | | | | | | | | | | | | internally. The load location for a Skylark Aspect is specified via a PathFragment, for consistency with current non-Aspect Skylark loads. This should be a semantics-preserving change for users. In a subsequent CL, I'll change the Skylark syntax to allow load statements to use labels as well as paths, with the goal of eventually deprecating the latter. Also: - Removed the hack for handling relative loads in the prelude file. - Refactored some redundant functionality in PackageFunction and SkylarkImportLookupFunction for handling loads. - Removed the ability to put the BUILD file for the package containing a Skylark file under a different package root than the Skylark file itself. This functionality isn't currently used and is inconsistent with Blaze's handling of the package path elsewhere. - Added BUILD files to a number of tests that load Skylark files; this is consistent with the requirement that all Skylark files need to be part of some package. - Changed the constants used to set the location of the prelude file from paths to labels. -- MOS_MIGRATED_REVID=107741568
* Implement propagation along dependencies for Skylark aspects.Gravatar Dmitry Lomov2015-11-02
| | | | | -- MOS_MIGRATED_REVID=106694515
* Refactor cycle related skyvalues to use shared empty valueGravatar Michajlo Matijkiw2015-10-13
| | | | | | | | | | | Previously the cycle values (were supposed to) share a common abstract base class, however usage was inconsistent. Instead refactor to eliminate the specialized value classes, remove the abstract class, and share a common empty placeholder value, which should be useful for future/other current empty values. -- MOS_MIGRATED_REVID=105217399
* Track skylark import cycles in terms of PackageIdentifiersGravatar Michajlo Matijkiw2015-10-12
| | | | | | | | | | We don't have any strong guarantees around the stability of SkyKey#toString(), which we rely on to canonicalize cycles. We can reasonably expect PackageIdentifier#toString() to be stable enough though. We also hide away the cycle detection details from external code in this change. -- MOS_MIGRATED_REVID=105093095
* Intern PackageIdentifiers as a memory optimization.Gravatar Miguel Alcon Pinto2015-10-01
| | | | | -- MOS_MIGRATED_REVID=104403040
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Allow Skylark import lookup values to be computed inline in order to avoid ↵Gravatar Janak Ramakrishnan2015-09-24
| | | | | | | reifying them in the Skyframe graph. -- MOS_MIGRATED_REVID=103758591
* Batch SkylarkImportLookupValue calls instead of doing them serially. Also ↵Gravatar Janak Ramakrishnan2015-09-22
| | | | | | | throw errors more eagerly in SkylarkImportLookupFunction -- don't try to request deps if the ast is in error. -- MOS_MIGRATED_REVID=103607939
* Roll back using labels rather than PathFragments for skylark loads.Gravatar Michajlo Matijkiw2015-09-22
| | | | | | | RELNOTES: -- MOS_MIGRATED_REVID=103606693
* Use Labels, rather than PathFragments, to represent Skylark loads ↵Gravatar John Field2015-09-22
| | | | | | | | | | | | | | | | | | | internally. This should be a semantics-preserving change for users. In a subsequent CL, I'll change the Skylark syntax to allow load statements to use labels as well as paths, with the goal of eventually deprecating the latter. Also: - Removed the hack for handling relative loads in the prelude file. - Refactored some redundant functionality in PackageFunction and SkylarkImportLookupFunction for handling loads. - Removed the ability to put the BUILD file for the package containing a Skylark file under a different package root than the Skylark file itself. This functionality isn't currently used and is inconsistent with Blaze's handling of the package path elsewhere. - Added BUILD files to a number of tests that load Skylark files; this is consistent with the requirement that all Skylark files need to be part of some package. - Changed the constants used to set the location of the prelude file from paths to labels. -- MOS_MIGRATED_REVID=103567562
* Move Label from the lib.syntax to the lib.cmdline package so that:Gravatar Lukacs Berki2015-09-21
| | | | | | | | | | - Label parsing can be simplified - lib.syntax is only contains the code for Skylark and is reasonably independent from the problem domain of building things This change is mostly only changes to imports declarations. The rest is reversing the dependency between :cmdline and :syntax and moving a tiny amount of code between Printer and FilesetEntry and the addition of SkylarkPrintableValue that I couldn't be bothered to separate out into its own change. -- MOS_MIGRATED_REVID=103527877
* Parse the label syntax "@//a:b" so that eventually we can make that the ↵Gravatar Lukacs Berki2015-09-15
| | | | | | | | | syntax that means "refer to the main repository". There isn't an overarching plan for what we are going to do with the cmdline package, which seems to be separated from the .syntax one in all sorts of awkward ways. -- MOS_MIGRATED_REVID=103088960
* Refactor Skylark Environment-sGravatar Francois-Rene Rideau2015-09-11
| | | | | | | | | | | | | | | | | | | | Make Environment-s freezable: Introduce a class Mutability as a revokable capability to mutate objects in an Environment. For now, only Environment-s carry this capability. Make sure that every Mutability is revoked in the same function that create... This reinstates a change that previously rolled-back because it broke the serializability of SkylarkLookupValue. Bad news: serializing it succeeds for the wrong reason, because a SkylarkEnvironment was stored as a result (now an Environment.Extension) that was Serializable but inherited its bindings from an Environment (now an Environment.BaseExtension) which wasn't Serializable. Apparently, Java doesn't try to serialize the bindings then (or at least doesn't error out when it fails), because these bindings map variable names to pretty arbitrary objects, and a lot of those we find in practice aren't Serializable. Thus the current code passes the same tests as the previous code, but obviously the serialization is just as ineffective as it used to be. -- MOS_MIGRATED_REVID=102776694
* Rollback of commit 5a94e59f02833f9142bad9203acd72626b089535.Gravatar Janak Ramakrishnan2015-09-08
| | | | | | | | | *** Reason for rollback *** Breaks serialization of SkyValues. -- MOS_MIGRATED_REVID=102457225
* Refactor Skylark Environment-sGravatar Francois-Rene Rideau2015-09-08
| | | | | | | | | | | | | | | | | Make Environment-s freezable: Introduce a class Mutability as a revokable capability to mutate objects in an Environment. For now, only Environment-s carry this capability. Make sure that every Mutability is revoked in the same function that creates it, so no Environment is left open for modification after being created and exported; exceptions for tests, the shell and initialization contexts. Unify Environment, SkylarkEnvironment and EvaluationContext into Environment. Have a notion of Frame for the bindings + parent + mutability. Replace the updateAndPropagate mechanism by a dynamicFrame. Simplify ValidationEnvironment, that is now always deduced from the Environment. -- MOS_MIGRATED_REVID=102363438
* Fix some warnings.Gravatar Ulf Adams2015-09-04
| | | | | -- MOS_MIGRATED_REVID=102332437
* Inject the prelude path through the rule class provider.Gravatar Ulf Adams2015-09-01
| | | | | -- MOS_MIGRATED_REVID=102020499
* Move global objects to RuntimeGravatar Francois-Rene Rideau2015-08-31
| | | | | | | | Move away global constants and global namespaces out of Environment and into a new file Runtime. -- MOS_MIGRATED_REVID=101940218
* Move PackageIdentifier to cmdlineGravatar Kristina Chodorow2015-08-31
| | | | | | | | This is necessary to have TargetResolver depend on it without making it depend on the packages target. First step of #389. -- MOS_MIGRATED_REVID=101790345
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-13 12:17:08 +0000