aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/rules/repository
Commit message (Collapse)AuthorAge
* Remove special handling of name attribute. Fixes #278Gravatar Googler2017-02-14
| | | | | | | | | | | | | | | | | | | | | | | | | The name attribute gets special treatment in the codebase, in that it's not simply yet another attribute but stored in it's own field. Thus, every callside dealing with attributes needs to be aware of this special case and explicitly handle the name attribute. It's easy to see that this can lead to bugs. For example, querying for the name attribute is currently broken due the querying code not being aware of the special case [1]. Discussions with experienced bazel developers came to the conclusion that there is no need (anymore) to treat the name attribute specially and thus we decided it's best to remove the special treatment and handle the name attribute as any other attribute. This change removes the handling of name attributes and also adds a test case to verify that bug [1] is fixed. [1] https://github.com/bazelbuild/bazel/issues/278 -- PiperOrigin-RevId: 147446345 MOS_MIGRATED_REVID=147446345
* Add environ attribute to repository_rule Gravatar Damien Martin-Guillerez2017-02-14
| | | | | | | | | | | | | | | | | | | This environ attribute let the repository rule implementor describes on which environment variable change a repository should be refetched. RELNOTES[NEW]: environ parameter to the repository_rule function let defines a list of environment variables for which a change of value will trigger a repository refetching. Design doc: https://bazel.build/designs/2016/10/18/repository-invalidation.html [step 3 & 4] Fixes #1595. -- Change-Id: Ibc2f93e69cb08baf86107cc9a9428b7a0eba1bac Reviewed-on: https://cr.bazel.build/8139 PiperOrigin-RevId: 147345203 MOS_MIGRATED_REVID=147345203
* Add dependency for external files to the corresponding RepositoryDirectoryValue Gravatar Damien Martin-Guillerez2017-01-26
| | | | | | | | | | | | | | | | Previously we were adding a dependency to the rule itself, however the repository itself can depends on further dependencies (environment, template files, ...) with Skylark repositories. Not invalidating when those change was causing weird invalidation issues that were shown with the invalidation on environment (basically a refetch of the repository on environment invalidation would not cause an invalidation of the corresponding package, the next call would not have invalidation of the repository but would see the change in the build files...). -- Change-Id: I8945d9885e6390734ba02ccc6c3c6ca639fcec35 Reviewed-on: https://cr.bazel.build/8137 PiperOrigin-RevId: 145675258 MOS_MIGRATED_REVID=145675258
* Inject the digest of the marker file in the RepositoryDirectoryValue Gravatar Damien Martin-Guillerez2017-01-19
| | | | | | | | | | | This digest make sure that we produce a different RepositoryDirectoryValue for different marker data, thus invalidating the node. -- Change-Id: Ibc6286c76de7a8d07ece2bb8125bfb75095f7784 Reviewed-on: https://cr.bazel.build/8136 PiperOrigin-RevId: 144948233 MOS_MIGRATED_REVID=144948233
* Remote repositories: add the infrastructure for extending the marker file Gravatar Damien Martin-Guillerez2017-01-17
| | | | | | | | | | | | | | | This add a markerData map to the RepositoryFunction#fetch function so RepositoryFunction-s can declare extraneous data to add to the marker file. The RepositoryFunction#verifyMarkerData is called to verify those data in order to know if the repository is up to date and need re-fetching. Design doc: https://bazel.build/designs/2016/10/18/repository-invalidation.html [step 2] -- Change-Id: I9083fb72a0142f418a7296f889cd3eaf32e92498 Reviewed-on: https://cr.bazel.build/7973 PiperOrigin-RevId: 144728497 MOS_MIGRATED_REVID=144728497
* Skylark repositories: propage value from --action_env to repository_ctx.environ Gravatar Damien Martin-Guillerez2017-01-10
| | | | | | | | | | | | | | | | | | | | | | The environment is now computed with a mixture of the client environment and the values specified by the --action_env flag. If a user want to overwrite its environment for skylark repository, they can do `--action_env FOO=BAR` and the repository will see FOO as having the value BAR, whichever value is set in the client environment. Also propagate it to all repository functions, and deduplicate the way the client environment is passed to repository functions. Design doc: https://bazel.build/designs/2016/10/18/repository-invalidation.html [step 1] RELNOTES[INC]: repository_ctx environment is now affected by --action_env flag (value from the client environment will be replaced by value given on the command line through --action_env). -- Change-Id: I131a9695439aa9949d5001f820e2ae450e41332f Reviewed-on: https://cr.bazel.build/7971 PiperOrigin-RevId: 144091492 MOS_MIGRATED_REVID=144091492
* Revert "Revert "Fix repositories to use 'BUILD.bazel' as the name for the ↵Gravatar John Cater2016-12-21
| | | | | | | | | | | | symlinked/new build file, which has less changes of causing conflicts. Fixes #2226."" This reverts commit fa3f55a8785c28253b2f836ddc84e994e48edd7f. -- Change-Id: Ife393ebf4966cfaf2a487c80c67a781ca7ee69b0 Reviewed-on: https://cr.bazel.build/8051 PiperOrigin-RevId: 142555911 MOS_MIGRATED_REVID=142555911
* Revert "Fix repositories to use 'BUILD.bazel' as the name for the ↵Gravatar John Cater2016-12-16
| | | | | | | | | | | | | symlinked/new build file, which has less changes of causing conflicts. Fixes #2226." This reverts commit 2f31d60d1454cd324196a5159262c1ba32eabda8. Re-open #2226. -- Change-Id: I7f223df4bb239daf8c306955e92f817310e6cd4f Reviewed-on: https://cr.bazel.build/7950 PiperOrigin-RevId: 142257579 MOS_MIGRATED_REVID=142257579
* Fix repositories to use 'BUILD.bazel' as the name for the symlinked/new ↵Gravatar John Cater2016-12-13
| | | | | | | | | | build file, which has less changes of causing conflicts. Fixes #2226. -- Change-Id: I4a378d693004d4d6857c78d7ffe886ada3fb9239 Reviewed-on: https://cr.bazel.build/7850 PiperOrigin-RevId: 141909918 MOS_MIGRATED_REVID=141909918
* Fix private visibility for aliased targets. Gravatar Lukacs Berki2016-12-06
| | | | | | | | | | Also a drive-by improvement on some related error messages. RELNOTES[INC]: Only targets with public visibility can be bound to something in //external: . -- PiperOrigin-RevId: 141178039 MOS_MIGRATED_REVID=141178039
* Support multiple mirror URLs for external reposGravatar Justine Tunney2016-11-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change improves upon 4c67807964e37cfd55bbcda4c6374fcc480bcecc. - A urls attribute has been added to the native workspace rules, with the exception of maven_jar and git_repository. The Skylark repository API also supports multiple URLs now. - The earlier mirrors in the list are preferred. Failover will happen automatically in parallel. - The first 32kB of data is checked before choosing a mirror in order to evade captive portals. - If one's Internet goes down or a download times out, then the download will resume automatically where it left off, provided the server supports RFC7233 for that particular file. Please note that GitHub does not support this for archive snapshots. Files should always be mirrored to a CDN, e.g. GCS, because they support this. - A semaphore is now used on downloads so only 8 can happen at once. Fixes #1814 Fixes #2131 Fixes #2008 Fixes #1968 Fixes #1717 Fixes #943 Wont fix #1194 Fixes tensorflow/tensorflow#5933 Fixes tensorflow/tensorflow#5924 Fixes tensorflow/tensorflow#5924 Fixes tensorflow/tensorflow#5432 See #1607 See #821 See tensorflow/tensorflow#5080 See tensorflow/tensorflow#5029 See tensorflow/tensorflow#4583 See tensorflow/tensorflow#4058 RELNOTES: A urls attribute has been added to repository rules to support multiple mirror URLs for reliably downloading files. -- MOS_MIGRATED_REVID=140495736
* Adds a check that the new local repository path exists and is aGravatar John Cater2016-10-25
| | | | | | | | | | | directory. Fixes #1981. -- Change-Id: I16a96be3f4f9de66e608b1914a2554613ddc096a Reviewed-on: https://bazel-review.googlesource.com/c/6910/ MOS_MIGRATED_REVID=137192543
* Allow new_ rules to overwrited BUILD files in downloaded reposGravatar Kristina Chodorow2016-09-28
| | | | | | | | | | | My previous change carefully checked that the file was a symlink before removing it and added a test with local repositories... and it of course isn't a symlink with downloaded repositories and crashes. Fixes #1697. -- MOS_MIGRATED_REVID=134536130
* Fix build_file_content overwriting source filesGravatar Kristina Chodorow2016-09-14
| | | | | | | Fixes #1697. -- MOS_MIGRATED_REVID=133056813
* Don't print stack traces when repository rules have the wrong typeGravatar Kristina Chodorow2016-09-12
| | | | | | | | | This also entirely disallows select() in repository rules. All repository rules should now error out if the wrong type is given, instead of printing a stack trace. Fixes #1307. -- MOS_MIGRATED_REVID=132872804
* Make the code that is using execRoot paths use execRoot pathsGravatar Kristina Chodorow2016-09-09
| | | | | | | | | | | Chipping away at making the big CL for #1262 smaller. Only runfiles paths are different right now, so this makes getPathUnderExecRoot and getSourceRoot return the same thing. Also corrected a couple places where Label.EXTERNAL_PATH_PREFIX and Label.EXTERNAL_PACKAGE_NAME were being used incorrectly. -- MOS_MIGRATED_REVID=132671081
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Add an enum representing the specific build file name (WORKSPACE, BUILD) to ↵Gravatar John Cater2016-08-03
| | | | | | | the PackageLookupValue to reduce the number of references to the filename "BUILD". -- MOS_MIGRATED_REVID=129203257
* Remove not-quite necessary serialization bitsGravatar Michajlo Matijkiw2016-07-15
| | | | | | | What we really are doing here is formatting. -- MOS_MIGRATED_REVID=127481183
* Make repository name warning less noisyGravatar Kristina Chodorow2016-07-01
| | | | | | | This way it won't print if the repo maintainer doesn't set the repo name. -- MOS_MIGRATED_REVID=126300205
* Rollback of commit bdfd58a8ca2ed5735d6aaa5b238fb0f689515724.Gravatar Laurent Le Brun2016-06-20
| | | | | -- MOS_MIGRATED_REVID=125160288
* Make the execution root match the runfiles tree structure for external ↵Gravatar Kristina Chodorow2016-06-17
| | | | | | | | | | | | | | | | | | | | | repositories One interesting side effect of how this is implemented is that for external repositories, bin/ and genfiles/ are combined. External repo output is under bazel-out/local-fastbuild/repo_name for each repo. Fixes #1262. RELNOTES[INC]: Previously, an external repository would be symlinked into the execution root at execroot/local_repo/external/remote_repo. This changes it to be at execroot/remote_repo. This may break genrules/Skylark actions that hardcode execution root paths. If this causes breakages for you, ensure that genrules are using $(location :target) to access files and Skylark rules are using http://bazel.io/docs/skylark/lib/File.html's path, dirname, etc. functions. -- MOS_MIGRATED_REVID=125095799
* Create mark file for local repositories, tooGravatar Kristina Chodorow2016-06-17
| | | | | | | | | | | | | Local repositories were not marked, so if a WS file switched from remote->local->remote, on the first run the remote rule would create a mark file, on the second run the local rule would ignore it, and then on the third run the remote rule would look at the mark file and see, "I'm already up-to-date," leaving the repository as a local repo. Fixes #977. -- MOS_MIGRATED_REVID=125060180
* Allow use of Exceptions to exit early out of configured-target creation, ↵Gravatar Chris Parsons2016-05-24
| | | | | | | | | | instead of passing and checking null in all helpers. Demonstrates this pattern usage in a few select rules (e.g. AndroidBinary) where this was particularly egregious. There are many places which can benefit from this pattern -- this change doesn't try to fix them all at once. -- MOS_MIGRATED_REVID=123012378
* Make alias targets keep their own configuration.Gravatar Lukacs Berki2016-05-23
| | | | | | | | | This prevents the case of a rule acquiring the null configuration in case the alias points to an input file. Which, in turn, makes "bazel build" work for these targets. The reason it breaks is that TargetCompletionValue instances are created from the associated ConfiguredTarget in SkyframeExecutor#buildArtifacts(), which means that if the configurations do not match, TargetCompletionFunction requests a different ConfiguredTargetValue than it was created from. Fixes #1297. -- MOS_MIGRATED_REVID=122973526
* Invalidate external repositories when their names changeGravatar Brian Silverman2016-05-20
| | | | | | | | | | | | | | Without this, old repositories are still cached across the change to give a name in the WORKSPACE file, which results in annoying warnings. This should have been included in a7fea8a1467a89adde75e67bc99bf0e040f71fe9. Fixes #1166. -- Change-Id: I831b842dc07b4ca7925f003b68f4838d083b1166 Reviewed-on: https://bazel-review.googlesource.com/c/3660/ MOS_MIGRATED_REVID=122829553
* Update Bazel native rule documentation to use new ${link} syntax.Gravatar David Chen2016-05-20
| | | | | -- MOS_MIGRATED_REVID=122772973
* Add an "alias" rule.Gravatar Lukacs Berki2016-05-10
| | | | | | | This will be used to replace RedirectChaser so that we don't need to load packages during configuration creation anymore. -- MOS_MIGRATED_REVID=121935989
* Check for lower-case versions of HTTP(S)_PROXYGravatar Kristina Chodorow2016-05-05
| | | | | | | | | | wget accepts http_proxy and https_proxy. RELNOTES: Proxy settings can now be specified in http_proxy and https_proxy environment variables (not just HTTP_PROXY and HTTPS_PROXY). -- MOS_MIGRATED_REVID=121594117
* Set workspace name for http_file, http_jar, and maven_jarGravatar Kristina Chodorow2016-05-03
| | | | | | | | | | | | | | | | Fixes #1166. While I was here, I also: * Renamed the implementations of Decompressor to show that they are Decompressors, not SkyFunctions (they used to be, it's confusing to have them still have the -Function suffix). * Added a unit test and moved the testing target to the rule/repository subdirectory, since it was a good chance to break off a tiny chunk of the monolithic BUILD file in lib/. -- MOS_MIGRATED_REVID=121306165
* Check for additions to the directory in new_local_repositoryGravatar Kristina Chodorow2016-04-22
| | | | | | | | | | Fixes #806. RELNOTES: External repository correctness fix: adding a new file/directory as a child of a new_local_repository is now noticed. -- MOS_MIGRATED_REVID=120557511
* Make aspects work through bind(). Gravatar Lukacs Berki2016-04-22
| | | | | | | | | | | bind() is assumed to be able to provide any provider. This is suboptimal, but beats the alternative of traversing the dependency graph to an arbitrary depth. The reason for the removal of the iteration ability in TransitiveInfoCollection is that now aspects can be attached to BindConfiguredTarget, too, which is not a RuleConfiguredTarget. Whereas I could have implemented the iterator, it was used only in BindConfiguredTarget anyway, so there didn't seem to be much reason to. Some work towards #952. -- MOS_MIGRATED_REVID=120549877
* Add a workspace name to Skylark repositoriesGravatar Kristina Chodorow2016-04-21
| | | | | | | | | This prevents an ugly error when the main repository name is set. Fixes #1166. -- MOS_MIGRATED_REVID=120352395
* Overwrite the WORKSPACE file for new_local_repositoryGravatar Kristina Chodorow2016-04-20
| | | | | | | | | | | | | This prevents an ugly warning when you do something like: new_local_repository( name = "bazel_tools", path = ".", build_file = "BUILD", ) -- MOS_MIGRATED_REVID=120271366
* Remove the directories field from RepositoryDelegatorFunction.Gravatar Ulf Adams2016-04-15
| | | | | -- MOS_MIGRATED_REVID=119942803
* Make local_repository's interpretation of 'path' match the other rules'Gravatar Kristina Chodorow2016-04-01
| | | | | | | | | Noticed this while messing with remote runfiles. RELNOTES: local_repository can take a relative path. -- MOS_MIGRATED_REVID=118780775
* Support labels in new_{http,local}_repository's build_file attributeGravatar Damien Martin-Guillerez2016-03-31
| | | | | | | Fixes #855. -- MOS_MIGRATED_REVID=118711400
* Remove the directories field from RepositoryFunction.Gravatar Ulf Adams2016-03-31
| | | | | | | | This is problematic for coming up with a less stateful BlazeModule API, which - in turn - is a requirement for running multiple commands in the same server. -- MOS_MIGRATED_REVID=118696892
* Correctly reports error in the workspace fileGravatar Damien Martin-Guillerez2016-03-23
| | | | | | | | The previous implementation was hiding errors from before a load statement as a result of the workspace split. -- MOS_MIGRATED_REVID=117933781
* Use static creation method for SkyKey. This allows interning SkyKeys as they ↵Gravatar Janak Ramakrishnan2016-03-03
| | | | | | | are created, as opposed to when they are requested from the ParallelEvaluator. That delay can lead to large memory spikes and churn. -- MOS_MIGRATED_REVID=116224565
* Testing correct invalidation of Skylark Remote RepositoriesGravatar Damien Martin-Guillerez2016-02-25
| | | | | | | | | | | | | | | | | | A Skylark remote repository should be invalidated only when the WORKSPACE file change, or one of its dependency or the Skylark file change. This change include two fixes: - The path of the RepositoryDirectoryValue was incorrect when the directory root is a symlink and the repository is not local (and not refetching). This was never triggered before because the only rule that were symlinking their root were the local one. - Directories were unitialized for the SkylarkRepositoryFunction (was forgotten as part of a refactor when introducing it). -- MOS_MIGRATED_REVID=115547540
* Introduce SkylarkRepositoryModuleGravatar Damien Martin-Guillerez2016-02-17
| | | | | | | | | | | | | | | | | The SkylarkRepositoryModule declare the `repository_rule` function to Skylark to define new remote repository types (http://goo.gl/OZV3o0). The work is delagated to the `SkylarkRepositoryFunction` by the `RepositoryDelegatorFunction`. `SkylarkRepositoryContext` defines the `ctx` object passed to the `repository_rule` implementation function. This change also introduce a `SkylarkPath` and the necessary methods in `SkylarkRepositoryContext` to showcase the creation of a `local_repository` like repository. Issue #893: step 3 of the roadmap http://goo.gl/OZV3o0. -- MOS_MIGRATED_REVID=114895003
* Prevent load statements in remote repository's WORKSPACE file to breakGravatar Damien Martin-Guillerez2016-02-12
| | | | | | | | | | | | | | | If a load statements was present in a remote repository's WORKSPACE file, the parsing of the WORKSPACE file to get the workspace name will break the build because it tries to load the Skylark extension from the main repository. This change replace the parsing of the whole remote repository's WORKSPACE file by a parsing of only it first chunk (before the first load statement). This change also enforce that the workpace() function to be called only from the top of the WORKSPACE file. -- MOS_MIGRATED_REVID=114528640
* Adds a build_file_content attribute to new_git_repository, new_http_archive, andGravatar Alex Humesky2016-02-12
| | | | | | | | | | | | | new_local_repository which allows the build file to be specified by a string directly in the rule rather than using a separate file. build_file and build_file_content are both optional, but one or the other must be specified. RELNOTES: build_file_content attribute added to new_git_repository, new_http_archive, and new_local_repository. -- MOS_MIGRATED_REVID=114490435
* Make local_repository and new_local_repository work on Windows.Gravatar Lukacs Berki2016-02-11
| | | | | | | This makes it possible to compile //third_party/ijar with a bootstrapped Bazel on Windows in dslomov's tree. -- MOS_MIGRATED_REVID=114428109
* Fix extra spacing and grammar in error message.Gravatar Julio Merino2016-02-11
| | | | | -- MOS_MIGRATED_REVID=114344238
* Split the execution of the WORKSPACE file after each load statementGravatar Damien Martin-Guillerez2016-02-10
| | | | | | | | | | | | This is the main step toward supporting load statement of remote repository in the WORKSPACE file (bug #824). Load statement that refers to a remote repository will depend on the previous fragment of the workspace file. Issue #824 Step 3. -- MOS_MIGRATED_REVID=114237027
* Add an intermediate SkyFunction for resolving the external packageGravatar Damien Martin-Guillerez2016-02-07
| | | | | | | | | | | | | | The WORKSPACE file parsing needs to be separated into several parts to enable load of labels in the WORKSPACE file. This change adds an intermediate SkyFunction, ExternalPackageFunction, that requires all the WORKSPACE file part to be parsed to resolve //external: labels. Issue #824 Step 1. -- MOS_MIGRATED_REVID=113984026
* Parse the workspace name when a repository is loadedGravatar Kristina Chodorow2016-01-29
| | | | | | | | | Moved RepositoryValue to RepositoryDirectoryValue so that it could be cached (and not re-downloaded) even if the WorkspaceAST caused a Skyframe restart (as mentioned in https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/rules/repository/RepositoryDelegatorFunction.java#L130-L133). -- MOS_MIGRATED_REVID=113358489
* Remove unused ${SYNOPSIS} variable from Bazel native rule inline documentation.Gravatar David Chen2016-01-28
| | | | | | | | | | This variable is no longer used because the docgen template directly calls RuleDocumentationAttribte.getSynopsis() instead of using the SYNOPSIS variable, and RuleDocumentationAttribute.getHtmlDocumentation() resolves the SYNOPSIS variable to the empty string. -- MOS_MIGRATED_REVID=113249993
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-03 05:22:34 +0000