aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/test/java/com/google/devtools/build/lib
Commit message (Collapse)AuthorAge
* Implement flow control for gRPC and cleanup the interface to it a bit.Gravatar Lukacs Berki2016-09-06
| | | | | -- MOS_MIGRATED_REVID=132039389
* Skylark: Replaced late bound attributes with computed default attributesGravatar Mark Schaller2016-09-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Motivation: Compared to computed default attributes, late bound attributes are evaluated in a later phase (analysis instead of loading phase). While both mechanisms provide access to other attributes of a rule, only late bound attributes additionally provide access to the build configuration. However, late bound attributes could be used to return new labels that have not been loaded before. Since this happens in the analysis phase, it can break one of Blaze's underlying principles, thus introducing a serious correctness bug. We decided to replace late bound attributes in Skylark with computed default attributes since this moves the evaluation of values into the loading phase, thus fixing this bug. Moreover, none of the existing users of this mechanism required access to the build configuration, which means that the user impact of this change is minimal. Implementation details: Unlike attributes of non-Skylark rules, however, Skylark computed defaults need to be able to depend on more than two configurable attributes since all attributes of Skylark rules are configurable by default. This has two implications: 1. Unlike "normal" Skylark attributes, Skylark computed defaults need to know about the existence of other attributes in order to declare a dependency on them. This CL takes advantage of work previously done to require parameter names to be the names of attributes used by the computed default function. 2. Since Bazel computes the combinations of all possible attribute values, a Skylark rule with a computed default that depends on a lot of configurable attributes could crash Bazel. Consequently, this CL also introduces an upper bound (64) for the number of valid combinations. Caveats: 1. Getting the depended-on attributes' names from function paramters is mildly surprising. Alternatives: The best solution would be to keep SkylarkLateBound, but restrict it in a way that it can only return already loaded labels. This is not possible right now. -- MOS_MIGRATED_REVID=131967238
* Open-source many of tests from SkylarkIntegrationTest.Gravatar Dmitry Lomov2016-09-01
| | | | | -- MOS_MIGRATED_REVID=131929298
* First step towards allowing returning declared providers from rule ↵Gravatar Dmitry Lomov2016-09-01
| | | | | | | implementations. -- MOS_MIGRATED_REVID=131923497
* xcode_configure repository rule for determining locally installed versions ↵Gravatar Chris Parsons2016-09-01
| | | | | | | | | | | of xcode to select between, relaxing the requirement that users specify --xcode_version and --ios_sdk_version even if they have only one version installed. RELNOTES: On mac devices, detect locally installed versions of xcode to: 1. Use a sensible default if xcode is required but --xcode_version is unspecified. 2. Use sensible default iOS SDK version for the targeted version of xcode if ios_sdk_version is unspecified. -- MOS_MIGRATED_REVID=131841587
* Remove the "cc_*.includes attribute must refer to the third_party directory" ↵Gravatar Lukacs Berki2016-08-31
| | | | | | | | | | | warning from Bazel. This behavior doesn't really make sense except within Google. Fixes #1286. -- MOS_MIGRATED_REVID=131813322
* Add a couple more stubs for new resource_* toolsGravatar Googler2016-08-31
| | | | | | | Add for consistency -- missed from the earlier CL -- MOS_MIGRATED_REVID=131735784
* Do not load the transitive closure of the Android SDK on every build.Gravatar Lukacs Berki2016-08-31
| | | | | | | This makes the sanity check dependent on the configuration fragments returning proper roots, but it's not that bad because it already depends on them returning the proper set of implicit labels and #getImplicitLabels() will go away soon anyway. -- MOS_MIGRATED_REVID=131705535
* Make aspects attached to aliases work when their actual= attribute is a select.Gravatar Lukacs Berki2016-08-31
| | | | | | | Fixes https://github.com/bazelbuild/e4b/issues/6 . -- MOS_MIGRATED_REVID=131698950
* Avoid building android R class ijar / header jarGravatar Googler2016-08-29
| | | | | | | | | | | | | | | | | | | | | | | | | | It is already ignored and the real jar used as the compileTime jar in AndroidCommon. It's only used by the ide-resolve output group. The real jar is pretty close to the header jar in terms of content (field inits deleted). Tradeoff: The real jar is ~33% bigger (w/ same compression level), vs save build time and space by skipping the hjar actions and artifacts. At android_binary level, the hjar action can take ~40 seconds in some cases. At the library level, the hjar time is pretty tiny. hjars are not compressed. We could compress the real jars to make up for the extra 30% size (and beat the hjar size), but that adds ~25% elapsed time to do the compression (e.g., 10s vs 7.5s). Compression can shrink the jar 2-3x (37MB vs 100MB). -- MOS_MIGRATED_REVID=131588283
* Add some type arguments to make Bazel compile again.Gravatar Lukacs Berki2016-08-29
| | | | | -- MOS_MIGRATED_REVID=131570943
* Replace doc pages with redirects to versioned doc pages.Gravatar David Chen2016-08-29
| | | | | | | | | | | | | | * Add a new `redirect` Jekyll layout. * Replace all pages under docs/ with redirects to corresponding page under versions/master/. * Prepend links on Documentation sidebar, including generated navs for the Skylark Library and Build Encyclopedia, with prefix for versioned directory. * Add code to both the internal jekyll-config.sh and external jekyll-tree.sh to add redirect pages for the Skylark Library and Build Encyclopedia. * Bring the branched User Manual doc up to date with latest changes. -- MOS_MIGRATED_REVID=131568800
* Add realpath method to SkylarkPath.Gravatar David Chen2016-08-29
| | | | | | | | | Fixes #1685 RELNOTES: Add path.realpath() method for Skylark repositories. -- MOS_MIGRATED_REVID=131568751
* Some little fixes to ResourceManager.Gravatar Philipp Wollermann2016-08-29
| | | | | | | | - Make sure that empty ResourceSets are always == ResourceSet.ZERO and use that for easier comparison. - No longer allow nested resource acquisition, because it may lead to deadlocks. -- MOS_MIGRATED_REVID=131567446
* Fix resource leakage on interrupt in ResourceManager identified by philwo.Gravatar Janak Ramakrishnan2016-08-27
| | | | | | | On interrupt, we never released any resources that we had asked to acquire, even though those resources would eventually be acquired. -- MOS_MIGRATED_REVID=131431321
* Remove all references to (incorrect) BlazeDirectories.buildDataDirectory rootGravatar Kristina Chodorow2016-08-26
| | | | | | | | | The buildDataDirectory is calculated off of the incorrect execroot. More progress towards #1681. -- MOS_MIGRATED_REVID=131407798
* sandbox: Allow network access for builds by default.Gravatar Philipp Wollermann2016-08-26
| | | | | | | | | | | | | This solves a performance issue that slowed down builds by about 40% at least on Linux, due to clone() with CLONE_NEWNET becoming extremely slow (>1 second) for highly parallel builds. See this thread for a discussion: https://lkml.org/lkml/2014/8/20/40 For the sake of consistency, we apply the same policy on OS X, too. If we find a better way to block network access for processes on Linux that doesn't have this performance hit, we will revisit this. RELNOTES: Sandboxed builds allow network access for builds by default. Tests will still be run without networking, unless "requires-network" is specified as a tag. -- MOS_MIGRATED_REVID=131393514
* --Gravatar Philipp Wollermann2016-08-26
| | | | MOS_MIGRATED_REVID=131383292
* Add client environment to ActionExecutionContextGravatar Klaus Aehlig2016-08-26
| | | | | | | | | | | As the execution of an action now also depends on the client environment, make the latter part of the ActionExecutionContext, so that enough context is provided to actually execute an action. -- Change-Id: Ida7bf407ef0c0375728faba92494bfd47dcbaeb8 Reviewed-on: https://bazel-review.googlesource.com/#/c/5391 MOS_MIGRATED_REVID=131377490
* Some optimizations in Type#flatten (used under the covers by ↵Gravatar Nathan Harmata2016-08-25
| | | | | | | | | | | | | AggregatingAttributeMapper#visitLabels): -Rename Type#flatten to Type#extractLabels. -Change the return type of Type#extractLabels from Collection to Iterable. This way we don't need to create and concatenate large lists. -Add an internal-only Type#containsLabels so this way ListType and DictType can have efficient implementations of Type#extractLabels. Note that AggregatingAttributeMapper#visitLabels is called multiple times on the same in several different places during the lifetime of a non-incremental Blaze invocation (e.g. during Package loading, during transitive target visitation, etc) -- MOS_MIGRATED_REVID=131311698
* Remove ArtifactFactory dependency on incorrect exec rootGravatar Kristina Chodorow2016-08-25
| | | | | | | | | | | Somewhat trickily, this changes the execRoot field from referring to [output_base]/execroot/wsname (not really the exec root) to [output_base]/execroot (actually the execroot). Progress on #1681. -- MOS_MIGRATED_REVID=131286181
* Action#discoverInputs returns an Iterable<Artifact> instead of aGravatar Cal Peyser2016-08-25
| | | | | | | Collection<Artifact> -- MOS_MIGRATED_REVID=131285541
* Bazel shell scripts: fix links in licence comments.Gravatar Laszlo Csomor2016-08-24
| | | | | | | I suspect they were overwritten by an eager sed "s,//,#,g" command. -- MOS_MIGRATED_REVID=131159373
* Extend Action interface by client variablesGravatar Klaus Aehlig2016-08-24
| | | | | | | | | | | | | | | | As per our design on [Specifying environment variables](http://bazel.io/designs/2016/06/21/environment.html), actions may depend, in a controlled way, on the environment in which the Bazel client is invoked. Those environment variables are considered essential for the action, in the sense that it was to be repeated if either of them changes their value (note that other variables in client environment may well change without invalidating actions). Therefore, make the variables that need to be taken from the client environment part of the meta data for actions. -- Change-Id: I2ff6cf40b4ce8e0fea5c7e464f5f3b3e693025ac Reviewed-on: https://bazel-review.googlesource.com/#/c/5390 MOS_MIGRATED_REVID=131150211
* Automated [] rollback of commit 846a5ab98fc26d72024890fdb79a5d3bc6a5a1ba + ↵Gravatar Nathan Harmata2016-08-23
| | | | | | | | | | | | | | | manual rollback of [] *** Reason for rollback *** Depot has been fixed / is in the process of being fixed. See the work tracked on [] *** Original change description *** Automated [] rollback of commit bb5d5efb4b50710241b5b374eb67084f4bf08278. -- MOS_MIGRATED_REVID=131095905
* Avoid copying SkyKeys into a set and then into a map when retrieving them. ↵Gravatar Janak Ramakrishnan2016-08-23
| | | | | | | | | | | | | Instead, just put them directly into a map. This avoids the memory churn and CPU cost of the set. As a result, we have to use HashMaps instead of ImmutableMap.Builders, which I hope is ok (especially since we're not keeping them around), and due to that, we have some nice nondeterminism in the returned order, which matters for some cycle-checking tests. Also, don't use a map at all when we don't need to (when building events). Note that, since we have to deduplicate at some point, this means that changing the return type of SkyFunction.Environment#getValues to not be a random-access map is probably not worth it. Changing the return type of ProcessableGraph#getBatch to not be a random access map might still be worthwhile, although it might require some funny operations. -- MOS_MIGRATED_REVID=131070418
* Prune .pcm files based on the results of include scanning.Gravatar Googler2016-08-23
| | | | | | | Basically, take the set of headers found by #include scanning and check what modules they are coming from. If a module provides at least one of the required headers, it is required as are all of its dependent modules (because of the way modules are implemented). Only use the actually required modules as compilation inputs and as flags handed in on the command line. Also move the logic to calculate top-level modules from the analysis phase into the execution phase. In the long run, we might be able to completely remove this logic now, but for now, we want to be able to quickly switch between the old and the new behavior. Thus, pruning of modules is now guarded on a feature prune_module_headers. -- MOS_MIGRATED_REVID=131058820
* More flexible LValue syntaxGravatar Vladimir Moskva2016-08-23
| | | | | -- MOS_MIGRATED_REVID=131056178
* Make Digest (renamed Md5Digest) a little more multi-purpose.Gravatar Shreya Bhattarai2016-08-23
| | | | | -- MOS_MIGRATED_REVID=130986194
* Remove CppCompilationContext.equals/hashCode. Making them throw an exceptionGravatar Googler2016-08-23
| | | | | | | | | shows that they aren't actually used anymore. If they were, they wouldn't work as expected as the nested sets that are used in the comparison don't implement hashCode or equals. -- MOS_MIGRATED_REVID=130945911
* Rollback of commit bb5d5efb4b50710241b5b374eb67084f4bf08278.Gravatar Tobias Werth2016-08-23
| | | | | -- MOS_MIGRATED_REVID=130941264
* Remove support for thin archives.Gravatar Lukacs Berki2016-08-23
| | | | | -- MOS_MIGRATED_REVID=130938527
* Move SkylarkImport from LoadStatement to BuildFileASTGravatar Miguel Alcon Pinto2016-08-22
| | | | | | | This allow us to skip the import validation in non-build usages. -- MOS_MIGRATED_REVID=130936612
* Instead of filtering for dynamic libraries when creating solib symlinks, ↵Gravatar Lukacs Berki2016-08-22
| | | | | | | | | assert that solib symlinks always point to dynamic libraries. Note that this makes it possible to crash Bazel with a malicious CROSSTOOL file, but since we are migrating to cc_toolchain_suite which raises a user-friendly error earlier, this is okay. -- MOS_MIGRATED_REVID=130915262
* Make Bazel not crash when a cc_library is in the hdrs of another cc_library.Gravatar Lukacs Berki2016-08-22
| | | | | -- MOS_MIGRATED_REVID=130914217
* Implement timeouts on Windows.Gravatar Lukacs Berki2016-08-22
| | | | | | | Makes #1664 much less acute. -- MOS_MIGRATED_REVID=130750731
* Fixes noneable annotation not being enforced in SkylarkCallable.Gravatar Pedro Liberal Fernandez2016-08-19
| | | | | -- MOS_MIGRATED_REVID=130665520
* Make BuildViewTest#testErrorBelowCycle properly deterministic so that it can ↵Gravatar Janak Ramakrishnan2016-08-19
| | | | | | | test what it's supposed to test, as pointed out by gregce. -- MOS_MIGRATED_REVID=130661645
* Allow CompileOneDependencyTransformer#transformCompileOneDependency to throw ↵Gravatar Janak Ramakrishnan2016-08-19
| | | | | | | InterruptedException instead of wrapping as a TargetParsingException. -- MOS_MIGRATED_REVID=130660310
* Implement the first stage of Bazel's "Sandbox 2.0" for Linux.Gravatar Philipp Wollermann2016-08-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This has the following improvements upon the older one: - Uses PID namespaces, PR_SET_PDEATHSIG and a number of other tricks for further process isolation and 100% reliable killing of child processes. - Uses clone() instead of unshare() to work around a Linux kernel bug that made creating a sandbox unreliable. - Instead of mounting a hardcoded list of paths + whatever you add with --sandbox_add_path, this sandbox instead mounts all of /, except for what you make inaccessible via --sandbox_block_path. This should solve the majority of "Sandboxing breaks my build, because my compiler is installed in /opt or /usr/local" issues that users have seen. - Instead of doing magic with bind mounts, we create a separate execroot for each process containing symlinks to the input files. This is simpler and gives more predictable performance. - Actually makes everything except the working directory read-only (fixes #1364). This means that a running process can no longer accidentally modify your source code (yay!). - Prevents a number of additional "attacks" or leaks, like accidentally inheriting file handles from the parent. - Simpler command-line interface. - We can provide the same semantics in a Mac OS X sandbox, which will come in a separate code review from yueg@. It has the following caveats / known issues: - The "fallback to /bin/bash on error" feature is gone, but now that the sandbox mounts everything by default, the main use-case for this is no longer needed. The following improvements are planned: - Use a FUSE filesystem if possible for the new execroot, instead of creating symlinks. - Mount a base image instead of "/". FAQ: Q: Why is mounting all of "/" okay, doesn't this make the whole sandbox useless? A: This is still a reasonable behavior, because the sandbox never tried to isolate your build from the operating system it runs in. Instead it is supposed to protect your data from a test running "rm -rf $HOME" and to make it difficult / impossible for actions to use input files that are not declared dependencies. For even more isolation the sandbox will support mounting a base image as its root in a future version (similar to Docker images). Q: Let's say my process-specific execroot contains a symlink to an input file "good.h", can't the process just resolve the symlink, strip off the file name and then look around in the workspace? A: Yes. Unfortunately we could not find any way on Linux to make a file appear in a different directory with *all* of the semantics we would like. The options investigated were: 1) Copying input files, which is much too slow. 2) Hard linking input files, which is fast, but doesn't work cross- filesystems and it's also not possible to make them read-only. 3) Bind mounts, which don't scale once you're up in the thousands of input files (across all actions) - it seems like the kernel has some non-linear performance behavior when the mount table grows too much, resulting in the mount syscall taking more time the more mounts you have. 4) FUSE filesystem, good in theory, but wasn't ready for the first iteration. RELNOTES: New sandboxing implementation for Linux in which all actions run in a separate execroot that contains input files as symlinks back to the originals in the workspace. The running action now has read-write access to its execroot and /tmp only and can no longer write in arbitrary other places in the file system. -- Change-Id: Ic91386fc92f8eef727ed6d22e6bd0f357d145063 Reviewed-on: https://bazel-review.googlesource.com/#/c/4053 MOS_MIGRATED_REVID=130638204
* Fixes order of keyword arguments in the error message printed when a JavaGravatar Pedro Liberal Fernandez2016-08-18
| | | | | | | | | method cannot be found when called from Skylark. PAIR=laurentlb -- MOS_MIGRATED_REVID=130636387
* Use array comparison and not reference comparison between digests.Gravatar Shreya Bhattarai2016-08-18
| | | | | -- MOS_MIGRATED_REVID=130576075
* Fix NPE when aspect propagates over an alias.Gravatar Googler2016-08-18
| | | | | -- MOS_MIGRATED_REVID=130553300
* RELNOTES: The string list returned by the skylark 'glob' function is now ↵Gravatar Nathan Harmata2016-08-18
| | | | | | | | | | | | | | sorted. Previously, it would return a list formed by concatenating the sorted results of each pattern in the 'includes' list. A bunch of cleanups and one bug fix: -Remove the unused-except-for tests GlobCache#globsUpToDate. This code has been dead for a very very long time, ever since we switched to using Skyframe. -Change the semantics of the 'glob' function as described above. -Change UnixGlob to return unsorted results. Document this in UnixGlob and GlobCache. -Change LegacyGlobber to conditionally return sorted results. Have users other than PackageFunction get sorted results (as described above). Have PackageFunction's use case get completely unsorted results, and have PackageFunction do the sorting itself. -Have PackageFunction's HybridGlobber unconditionally sort the glob result list. This ensure deterministic glob results, fixing a bug where the order of the elements of the result depended on the contents of the Skyframe graph, which of course depends on the sequence of incremental Blaze commands. -- MOS_MIGRATED_REVID=130540152
* Improve BlazeModule documentation.Gravatar Ulf Adams2016-08-18
| | | | | | | Also allow AbruptExitException from all server startup hooks. -- MOS_MIGRATED_REVID=130513167
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Have AQV propagate the most severe error encountered by any of its tasks. ↵Gravatar Nathan Harmata2016-08-12
| | | | | | | | | Make sure that ParallelEvaluator treats SchedulerExceptions as less severe than other RuntimeExceptions (it already did, but add a comment emphasizing this). The combination of the above means that we don't ignore hypothetical crashes in ParallelEvaluator worker threads _after_ a SchedulerException (e.g. due to an error in nokeep_going evaluation). -- MOS_MIGRATED_REVID=130044230
* Changes DependencyResolver <Attribute, Dep> map from a ListMultimap to new classGravatar Greg Estren2016-08-12
| | | | | | | | | | | | | | | | | | | | | | OrderedSetMultimap. This maintains insertion order while eliminating duplicates. Certain rules, in particular, otherwise break this invariant: https://github.com/bazelbuild/bazel/blo[]e3e28274cca5b87f48abe33884edb84016dd3/src/main/java/com/google/devtools/build/lib/skyframe/ConfiguredTargetFunction.java#L403 There's no reason (to my knowledge) to need multiple instances of the same <Attribute, Dependency> pair. More context from Google code review: (Michael Staib): > There are many things which pass around a dependentNodeMap or help construct one or modify one. We want an interface which has the right guarantees. > ListMultimap is not the right interface because it has no guarantee of unique elements, which we want - we don't want the problem that this CL ran into, and there's no reason (that we know of, to be confirmed) that anyone would want multiple identical Dependencies. > SetMultimap is not the right interface because it has no guarantee of deterministic iteration order or efficient iteration, which we want - dependency order sometimes matters (e.g., Java classpath or C++ link order). > We agreed that the best way to get what we want is to define our own interface with its own simultaneous uniqueness and iterability guarantees. Unspoken in the discussion was why we wouldn't just use LinkedHashMultimap as the thing we pass around. IMO the reason for that is that we don't care that it be a LinkedHashMultimap specifically; if tomorrow Guava comes out with a faster cooler map that has deterministic and efficient iteration and guarantees element uniqueness, we want it. > In this case we're going to make the "interface" be a (final?) class: OrderedSetMultimap, an extension of ForwardingSetMultimap which delegates to LinkedHashMultimap, an implementation which does support both of those guarantees. > I had mentioned in the conversation that none of the Multimap implementations make guarantees about key iteration order, but this is not true - LinkedHashMultimap preserves key insertion order. We should perhaps declare this as part of the OrderedSetMultimap contract as well. -- MOS_MIGRATED_REVID=130037643
* Rollback of commit f107debac45ddf5859b1eb963379769b5815b18f. Also includes ↵Gravatar Janak Ramakrishnan2016-08-12
| | | | | | | | | the logical rollback of commit 67ad82a319ff8959e69e774e7c15d3af904ec23d. RELNOTES[INC]: Bazel supports Unix domain sockets for communication between its client and server again, temporarily, while we diagnose a memory leak. -- MOS_MIGRATED_REVID=130027009
* Roll forward fix of global var shadow detectionGravatar Jon Brandvein2016-08-12
| | | | | | | RELNOTES[INC]: Skylark: It is an error to shadow a global variable with a local variable after the global has already been accessed in the function. -- MOS_MIGRATED_REVID=130014492
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-05 23:30:37 +0000