aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/skyframe/packages
Commit message (Collapse)AuthorAge
* Filter out events from analysis when constructing execution-phase values in ↵Gravatar janakr2018-08-14
| | | | | | | | | | Skyframe. This avoids some unnecessary iteration over already-emitted events that can show up in profiles, and allows us to store execution-phase values a bit more compactly, since we don't need to carry around wrapper objects and nested sets everywhere. This crucially depends on the fact that we can't build a target in the execution phase without first having analyzed it in a separate Skyframe call. Skyframe normally propagates all events/posts up the graph because it must be able to emit them if a user requests a node that only transitively depends on the node that emitted an event. However, because we do analysis in a separate Skyframe call, any warnings/posts associated with the analysis nodes will be emitted then, and we don't need to propagate them into execution. PiperOrigin-RevId: 208767078
* Introduce option flag experimental_enable_tools_defaults_package.Gravatar dbabkin2018-07-23
| | | | | | | | Default value is true, and behavior related to //tools/defaults package is not changed. If set it to false, then in-memory Dfaultpacked will not be created. RELNOTES:none PiperOrigin-RevId: 205643628
* Support unconditional fetching of repositoriesGravatar Klaus Aehlig2018-06-14
| | | | | | | | | | | Make all external repositories depend on an additional SkyValue controllable via commands, so support unconditional fetching of all external repositories, as it is needed by the the `sync` command. Improves on #5175, provides a work around for #4907. Change-Id: I30033614c1a2fad3f1363b85ff69cf92f697c255 PiperOrigin-RevId: 200543985
* Refactoring: makes the code simpler by deleting Metadata and InputArtifactData.Gravatar shahan2018-06-06
| | | | | | (minor) ActionFS now implements MetadataProvider.getInput PiperOrigin-RevId: 199575194
* Process 'repo_mapping' attribute from WORKSPACE rules. 'repo_mapping' is a ↵Gravatar dannark2018-06-04
| | | | | | | | | | | | | | | | way to remap references to repositories within an external repository by another name. This CL only adds the mappings to the Package object, but it does not actually enable the reassignments. Example usage (in a WORKSPACE file): local_repository( name = ?a?, path = ?../a?, repo_mapping = {?@x? : ?@y?} ) This change also creates a new SkyKey which represents the mappings. This is to prevent all packages from depending on the external package, and instead depending just on the mappings. i.e. a change to the WORKSPACE file that does not touch the mappings shouldn't cause a reload of the package. RELNOTES:None PiperOrigin-RevId: 199187963
* Use the local JDK as the default target javabaseGravatar cushon2018-05-14
| | | | | | and continue to use the embedded JDK as the default host_javabase. PiperOrigin-RevId: 196471714
* Allow SkyFunctions to return a sentinel value indicating that all of a ↵Gravatar janakr2018-04-30
| | | | | | | | | | node's in-progress data should be forgotten, and its evaluation should be restarted from scratch, as if it were freshly created/dirtied. To guard against this happening unexpectedly, any such events are passed to a GraphInconsistencyReceiver, which can verify that the SkyFunction is behaving properly. This is the first change in a series to permit action rewinding when it is discovered that a previously generated input file is no longer available. When an action detects that one of its inputs is unusable, it can return this sentinel value, causing it to be re-evaluated from scratch. Follow-up changes will make the node corresponding to the input, and the node corresponding to the action that generated the input, dirty when this happens, causing the upstream action to be re-run, regenerating the desired input. Currently works for builds that do not keep edges, although follow-ups may make this possible for all builds. PiperOrigin-RevId: 194863097
* Allow setting the value which BazelPackageLoader returns for ↵Gravatar carmi2018-03-29
| | | | | | | | | native.bazel_version. This allows it to work in workspaces that use https://github.com/bazelbuild/bazel-skylib/blob/a5e23fd4c0b766f38f80d5edf10073ce990b2fa1/lib/versions.bzl RELNOTES: None PiperOrigin-RevId: 190950709
* Fix test flakiness due to the filesystem mutations done during the 'fetch' ↵Gravatar nharmata2018-03-23
| | | | | | | | | work done by BazelPackageBuilderHelperForTesting#sanityCheckBazelPackageLoader. We fix this flakiness by _not_ having RepositoryDelegatorFunction unconditionally 'fetch' local repos. We then have to make a change to BazelPackageLoader test to reinstate these mutations, since it was only working before due to the unconditional filesystem mutations done here :p RELNOTES: None PiperOrigin-RevId: 190222830
* Victory lap: Remove all code that used to support the three-argument form of ↵Gravatar lberki2018-03-23
| | | | | | | vardef(). RELNOTES: None. PiperOrigin-RevId: 190196933
* Allow BazelPackageLoader to load external repositories.Gravatar carmi2018-03-13
| | | | | | | | | Also, disallow BazelPackageLoader from fetching missing external repos. Integration tests for BazelPackageLoader wrt external repos will be left for a follow-up CL. RELNOTES: None. PiperOrigin-RevId: 188967694
* Fix PackageFunction's call to Package.Builder.Helper#onLoadingComplete to passGravatar nharmata2018-03-07
| | | | | | | | | | along the wall time of the load, even when the package in question was in PackageFunction's internal cache (e.g. the current #compute call is a PackageFunction Skyframe restart). Also clarify the intent of the 'loadTimeMs' param in #onLoadingComplete. RELNOTES: None PiperOrigin-RevId: 188253198
* Expose --output_user_base to the Bazel server processGravatar Klaus Aehlig2018-02-27
| | | | | | | | | | | ...so that it can use that path to compute other directories in the output user base, in particular the default location for caches. The first cache we will add is the hash-index cache for downloads of external archives, but a spawn cache might be added later in the output user base as well. Change-Id: I24b1c33235c8f76ec008ecb1789163de2b2a45be PiperOrigin-RevId: 187164275
* A couple quality-of-life improvements for Bazel devs, in response to an ↵Gravatar nharmata2018-02-23
| | | | | | | | | | email from philwo@. (i) Only have TimestampGranularityMonitor log the first file of relevance. This reduces log spam, especially in tests, while still maintaining useful information in the logs. (ii) Don't have ExternalFilesHelper log the fact that it encountered an external file when we're in a unit test or an integration test. Tests, especially bazel tests that use external repositories, tend to involve lots of "external" files. RELNOTES: None PiperOrigin-RevId: 186799176
* Don't use null EvaluationProgressReceiver in MemoizingEvaluator (it was only ↵Gravatar janakr2018-01-30
| | | | | | null in AbstractPackageLoader and tests). PiperOrigin-RevId: 183896826
* Introduce Root class.Gravatar tomlu2018-01-17
| | | | | | | | | | | This class represents a root (such as a package path or an output root) used for file lookups and artifacts. It is meant to be as opaque as possible in order to hide the user's environment from sky keys and sky functions. Roots are used by RootedPaths and ArtifactRoots. This CL attempts to make the minimum number of modifications necessary to change RootedPath and ArtifactRoot to use these fields. Deprecated methods and invasive accessors are permitted to minimise the risk of any observable changes. RELNOTES: None PiperOrigin-RevId: 182271759
* Tell PackageFunction if it will never be used to do incremental package ↵Gravatar nharmata2017-12-22
| | | | | | | | | | | loading. Have PackageFunction optimize for the non-incremental case by not doing Skyframe [hybrid] globbing. Have AbstractPackageLoader use PackageFunction in non-incremental mode (recall that AbstractPackageLoader, by design, has no incrementality). Since AbstractPackageLoader now doesn't need GlobFunction, it now also doesn't need DirectoryListingFunction and therefore also doesn't need DirectoryListingStateFunction. In the entire Bazel codebase, DirectoryListingStateFunction is only used by DirectoryListingFunction. And DirectoryListingFunction's only use in package loading (in cases currently supported by AbstractPackageLoader*) is GlobFunction. *AndroidRepositoryFunction apparently uses DirectoryListingFunction, but AbstractPackageLoader doesn't yet support all Bazel repository types. We can address this in future changes, if needed. RELNOTES: None PiperOrigin-RevId: 179931359
* Remove some code leftover from back when BUILD file preprocessing was a ↵Gravatar nharmata2017-12-22
| | | | | | | thing. Also remove some TODOs that were leftover from before we added the caches used after a PackageFunction restart. RELNOTES: None PiperOrigin-RevId: 179926806
* Some performance improvements to AbstractPackageLoader.Gravatar nharmata2017-12-21
| | | | | | | | | (i) Share the syscall cache used for legacy globbing with the CachingPackageLocator also used by legacy globbing, otherwise it's not very "caching" :( (ii) Have the concurrency level of the syscall cache used by legacy globbing be set to the number of legacy globbing threads. This actually isn't a huge deal because of (i), since that removes a ton of duplicate cache lookups that would otherwise contend, but there's no reason not to do this. RELNOTES: None PiperOrigin-RevId: 179865599
* Enable syscall cache and eager sub-directory globbing in AbstractPackageLoader.Gravatar carmi2017-12-20
| | | | | RELNOTES: None PiperOrigin-RevId: 179757650
* Initialize the RuleClassProvider only once.Gravatar carmi2017-12-12
| | | | | | | This saves some CPU/wall time when creating multiple PackageLoader instances. RELNOTES: None PiperOrigin-RevId: 178834826
* Automated rollback of commit d0f06a6e4b61adc39bf5e1cfbae39501a89dc8e3.Gravatar nharmata2017-12-05
| | | | | RELNOTES: None PiperOrigin-RevId: 177965330
* Update PathPackageLocator to take a list of potential build file names,Gravatar John Cater2017-11-28
| | | | | | | | | | | | | | instead of assuming BUILD. - Default the list to the same value as PackageLookupFunction: BUILD.bazel, BUILD. - Move BuildFileNames to the packages package, so it is more generally available. Part of #4056. Change-Id: Ie12512b492cd7d47a9e56ec3bc209f829feaf4b5 PiperOrigin-RevId: 177261295
* Automated rollback of commit b64119807b014d9f3b99fb8a02e22daf1a8299b6.Gravatar nharmata2017-11-28
| | | | | | | | | | | | | *** Reason for rollback *** Tickles the shell integration test version of b/35042288 *** Original change description *** Change BlacklistedPackagesPrefixesFunction to take a pair of a hardcoded set of directories and a file path containing more directories to blacklist. The current usage of PrecomputedValue#BLACKLISTED_PACKAGE_PREFIXES_FILE is overly general and is only meaningfully used by unit tests; in practice, the blacklist file path can never change over the lifetime of the Bazel server. Perform a minor simplifying refactor as a result of this. RELNOTES: None. PiperOrigin-RevId: 177176068
* Change BlacklistedPackagesPrefixesFunction to take a pair of a hardcoded set ↵Gravatar nharmata2017-11-28
| | | | | | | of directories and a file path containing more directories to blacklist. The current usage of PrecomputedValue#BLACKLISTED_PACKAGE_PREFIXES_FILE is overly general and is only meaningfully used by unit tests; in practice, the blacklist file path can never change over the lifetime of the Bazel server. Perform a minor simplifying refactor as a result of this. RELNOTES: None. PiperOrigin-RevId: 177164057
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Allow/require callers of AbstractPackageLoader to set Skylark semantics ↵Gravatar brandjon2017-10-27
| | | | | | | | | | | explicitly Previously the default semantics were used unconditionally. Allowing non-default semantics is a feature. Requiring semantics to be specified explicitly helps to avoid unintentional divergence from the caller's intended semantics. We recently did the same thing for Skylark's Environment.Builder (https://github.com/bazelbuild/bazel/commit/b368b39f8ba1e8e8a67af50e5ade9127b2b149d7). Also pass Skylark semantics through Package.Builder.Helper, so that the extra verification done for shell tests uses the same semantics as the build. RELNOTES: None PiperOrigin-RevId: 173544885
* PackageLoader.loadPackages() tolerates duplicates in the list of packages to ↵Gravatar carmi2017-10-06
| | | | | | | load. RELNOTES: None PiperOrigin-RevId: 171298089
* Pass SkylarkSemantics through Skyframe instead of the options classGravatar brandjon2017-10-06
| | | | | | | Also remove the use of the @UsesOnlyCoreTypes annotation on SkylarkSemanticsOptions. It was only there to help mark that the options class was safe to put in Skyframe. RELNOTES: None PiperOrigin-RevId: 171248504
* Remove PrecomputedValue for BlazeDirectories. All consumers have the value ↵Gravatar janakr2017-09-25
| | | | | | | | injected directly, which makes sense, because it's immutable over the lifetime of the server. Step 3. PiperOrigin-RevId: 169717587
* Inject BlazeDirectories into RepositoryDelegatorFunction directly, instead ↵Gravatar janakr2017-09-25
| | | | | | | | of depending on a precomputed value. BlazeDirectories don't change over the lifetime of the Blaze server, and certainly not over the lifetime of a SkyframeExecutor instance, which already had a reference to them. Step 2. PiperOrigin-RevId: 169711443
* Automatic code cleanup.Gravatar cushon2017-09-15
| | | | PiperOrigin-RevId: 168802886
* Remove deepExecRoot from BlazeDirectories. It is now the default. Also ↵Gravatar janakr2017-09-14
| | | | | | inline constructor that did ServerDirectories creation inline: don't pollute production code for tests' convenience. PiperOrigin-RevId: 168652349
* More BUILD file refactorings.Gravatar philwo2017-09-06
| | | | | | | | | Split collect, concurrent, vfs, windows into package-level BUILD files. Move clock classes out of "util", into their own Java package. Move CompactHashSet into its own Java package to break a dependency cycle. Give nestedset and inmemoryfs their own package-level BUILD files. PiperOrigin-RevId: 167702127
* Fix BazelPackageLoader to handle local repositories, and clean up API byGravatar jcater2017-09-01
| | | | | | removing unused methods and making the Builder more general. PiperOrigin-RevId: 167150478
* Inline AttributeContainer.ATTRIBUTE_CONTAINER_FACTORY and ↵Gravatar laurentlb2017-07-03
| | | | | | | PathFragment.TO_PATH_FRAGMENT RELNOTES: None. PiperOrigin-RevId: 160668541
* Remove unused interface PreprocessorGravatar Klaus Aehlig2017-06-16
| | | | | Change-Id: I008c16d747c5d06f433af14cd0793536cff2723e PiperOrigin-RevId: 158975286
* Make PackageFunction's strategy for handling unreadable BUILD files ↵Gravatar nharmata2017-06-08
| | | | | | | configurable. Add a test for the current behavior of treating an unreadable BUILD file as a package loading error. RELNOTES: None PiperOrigin-RevId: 158314187
* Have BazelPackageLoader use the newly added BazelSkyframeExecutorConstants.Gravatar nharmata2017-06-07
| | | | PiperOrigin-RevId: 158159054
* Make PackageOrException.get() public + make loadPackages() variant.Gravatar carmi2017-05-23
| | | | | RELNOTES: None PiperOrigin-RevId: 156876531
* Create a loadPackages() method that loads multiple packages simultaneously, ↵Gravatar carmi2017-05-22
| | | | | | | | | using multiple threads. The immediate upside is that if multiple packages load the same bzl file, that file will only be read once when using loadPackages(). RELNOTES: None PiperOrigin-RevId: 156621988
* Make the number of legacy globbing threads configurable. PackageFactory's ↵Gravatar nharmata2017-05-19
| | | | | | | hardcoded default of 100 isn't a good choice for all PackageLoader users. RELNOTES: None PiperOrigin-RevId: 156468439
* PiperOrigin-RevId: 156331430Gravatar nharmata2017-05-19
|
* Automated g4 rollback of commit 9dec9a09d1e4d27e7cff73c440195a32d3b90752.Gravatar plf2017-05-05
| | | | PiperOrigin-RevId: 155194438
* Introduce BazelPackageLoader.Gravatar nharmata2017-05-04
This is a standalone library for loading packages. It is intended to be used by clients outside of Bazel (for now, Kythe). Perform BazelPackageLoader#loadPackage in the testing hook used in all of Bazel's unit tests and integration tests. This gives us very good test coverage for BazelPackageLoader. RELNOTES: None PiperOrigin-RevId: 155004070
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-23 11:57:26 +0000