aboutsummaryrefslogtreecommitdiffhomepage
path: root/src
Commit message (Collapse)AuthorAge
* remote: concurrent blob downloads. Fixes #5215Gravatar buchgr2018-06-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change introduces concurrent downloads of action outputs for remote caching/execution. So far, for an action we would download one output after the other which isn't as bad as it sounds as we would typically run dozens or hundreds of actions in parallel. However, for actions with a lot of outputs or graphs that allow limited parallelism we expect this change to positively impact performance. Note, that with this change the AbstractRemoteActionCache will attempt to always download all outputs concurrently. The actual parallelism is controlled by the underlying network transport. The gRPC transport currently enforces no limits on the concurrent calls, which should be fine given that all calls are multiplexed on a single network connection. The HTTP/1.1 transport also enforces no parallelism by default, but I have added the --remote_max_connections=INT flag which allows to specify an upper bound on the number of network connections to be open concurrently. I have introduced this flag as a defensive mechanism for users who's environment might enforce an upper bound on the number of open connections, as with this change its possible for the number of concurrently open connections to dramatically increase (from NumParallelActions to NumParallelActions * SumParallelActionOutputs). A side effect of this change is that it puts the infrastructure for retries and circuit breaking for the HttpBlobStore in place. RELNOTES: None PiperOrigin-RevId: 199005510
* Make tools in action inputs an error.Gravatar tomlu2018-06-02
| | | | | | | | | | Supporting tools in inputs introduces a slow linear scan. Such tools should be moved to the 'tools' attribute. If --incompatible_no_support_tools_in_action_inputs is set the inputs are scanned, but a helpful error message is issued to the user. Eventually we will remove the slow scanning. Errors will surface in the execution phase instead of during analysis, and the resulting error messages will be less obvious. RELNOTES: None RELNOTES[INC]: With --incompatible_no_support_tools_in_action_inputs enabled, Skylark action inputs are no longer scanned for tools. Move any such inputs to the newly introduced 'tools' attribute. PiperOrigin-RevId: 198996093
* Automated rollback of commit c4e128e2c6d8cacaeba034d6a3195796d50f1745.Gravatar tomlu2018-06-02
| | | | | | | Always generating source jars can lead to action conflicts. RELNOTES: None PiperOrigin-RevId: 198994272
* Removes most ActionInputFileCache functionality.Gravatar shahan2018-06-01
| | | | | | Actual class to be removed in a later change. PiperOrigin-RevId: 198937695
* Allow multiple trimming transition factories to be added.Gravatar mstaib2018-06-01
| | | | | | | | If multiple trimming transition factories are added, they are composed in the order they were added. RELNOTES: None. PiperOrigin-RevId: 198934666
* Amend a skylark test to attempt access of FilesToRunProvider.executableGravatar cparsons2018-06-01
| | | | | | | This serves a regression test of a previous breakage which broke SkylarkCallable method of FilesToRunProvider. RELNOTES: None. PiperOrigin-RevId: 198915920
* Migrate remaining java skylark types to skylarkbuildapiGravatar cparsons2018-06-01
| | | | | RELNOTES: None. PiperOrigin-RevId: 198911668
* Create a Bootstrap for config-related libraries of skylarkbuildapi.Gravatar cparsons2018-06-01
| | | | | RELNOTES: None. PiperOrigin-RevId: 198906931
* Migrate repository_rule to skylarkbuildapiGravatar cparsons2018-06-01
| | | | | RELNOTES: None. PiperOrigin-RevId: 198906068
* Android font resource supportGravatar Jingwen Chen2018-06-01
| | | | | | | | | | | | | | | | | | | | | Fixes #4381 Without layoutlib 26+, this test is expected to fail with: ``` ERROR: /private/var/tmp/_bazel_jin/ed227ac31d5e65f9c3effb1d1fe2605e/sandbox/4960637019899260266/execroot/io_bazel/_tmp/845372d54b28cdaedc66e35385e5009a/workspace.u0NdZr9w/java/bazel/BUILD:14:22: in resource_files attribute of android_binary rule //java/bazel:bin: 'java/bazel/res/font/Roboto.ttf' is not in the expected resource directory structure of <resource directory>/{anim,animator,color,drawable,interpolator,layout,menu,mipmap,raw,transition,values,xml}/<file> ERROR: Analysis of target '//java/bazel:bin' failed; build aborted: Analysis of target '//java/bazel:bin' failed; build aborted INFO: Elapsed time: 26.248s INFO: 0 processes. FAILED: Build did NOT complete successfully (36 packages loaded) ``` We will first need to upgrade the layoutlib jar in `//third_party/android_common` to 26+, rebase, then merge this set of tests. Closes #5287. Change-Id: Icad80e7e40df4ff6d2fc3373d37cfd9972e8d400 PiperOrigin-RevId: 198893525
* Mention string comparison in the documentationGravatar laurentlb2018-06-01
| | | | | RELNOTES: None. PiperOrigin-RevId: 198882702
* Migrate remaining apple-related skylark types to skylarkbuildapiGravatar cparsons2018-06-01
| | | | | RELNOTES: None. PiperOrigin-RevId: 198880301
* Use a new RecursivePackageProvider to ask skyframe for the correct package ↵Gravatar juliexxia2018-06-01
| | | | | | nodes in ConfiguredTargetQueryEnvironemnt. This lets us support recursive target patterns with cquery. PiperOrigin-RevId: 198879650
* Move TARGET_CPU from the CcToolchainProvider to BuildConfiguration.Gravatar jcater2018-06-01
| | | | PiperOrigin-RevId: 198877280
* [java_common.compile] Always generate a source jar.Gravatar elenairina2018-06-01
| | | | | | | | | | | java_common.compile doesn't generate the output source jar when a source jar is the only input for the compilation. This is wrong because the source jar can include APT generated sources. It is also inconsistent with java_library and leads to inconsistent Skylark rules where a declared output will not always have a generating action. RELNOTES: None. PiperOrigin-RevId: 198872718
* repository_resolved_file: pretty printGravatar Klaus Aehlig2018-06-01
| | | | | | | | As the list of external repositories with their resolved information can grow quite large, pretty print it, to make it easier for humans to inspect. Change-Id: I34b2d6df84e52c07471a0bfaf97994cabd45feff PiperOrigin-RevId: 198871353
* C++: Remove CcSpecificLinkParamsProvider.Gravatar plf2018-06-01
| | | | | | No longer needed since proto_library cannot be used for Java. PiperOrigin-RevId: 198859993
* Format all bzl files with buildifierGravatar vladmos2018-06-01
| | | | | | This CL aslo enables a presubmit check for correct formatting of all bzl files in //third_party/bazel. PiperOrigin-RevId: 198857055
* Windows,testing: add query regression testGravatar Laszlo Csomor2018-06-01
| | | | | | | | | | | This test could have prevented https://github.com/bazelbuild/bazel/issues/5300 Change-Id: I02e3b723958c81e0de80f2c2130b583ea4e7f40b Closes #5310. Change-Id: I02e3b723958c81e0de80f2c2130b583ea4e7f40b PiperOrigin-RevId: 198849437
* Converts PLACEHOLDER FileValues into RegularFileValue when queried viaGravatar shahan2018-05-31
| | | | | | ActionExecutionValue.getAllFileValues. PiperOrigin-RevId: 198807229
* Remove the Blaze strict java deps exemption for Android databinding. As ofGravatar Googler2018-05-31
| | | | | | | | | | | | | unknown commit, we are using the JavaBuilder processor exemption list instead of disabling strict deps for the affected rules outright. This narrows the degree to which the exemption is applied, and prevents other strict deps violations from accruing on targets that enable databinding. Step 2 of 2; we needed the change to roll out in JavaBuilder first to be able to fix up issues in the depot first before removing the exemption in Blaze. See https://github.com/bazelbuild/bazel/commit/7edc5d49c8e764ab2bdf19e3a9fb58ea30fa708f for the original attempt. PiperOrigin-RevId: 198793976
* Migrate remaining Android Skylark API to skylarkbuildapiGravatar asteinb2018-05-31
| | | | | | | | I don't know much about most of this code, but there should be no functional changes. RELNOTES: none PiperOrigin-RevId: 198773657
* Explicitly validate local resources are passed to Java compilation firstGravatar asteinb2018-05-31
| | | | | | | We thought we had a test for this, but it turns out we didn't. Add one now. RELNOTES: none PiperOrigin-RevId: 198772854
* Make test more explicit about what it is testing.Gravatar dannark2018-05-31
| | | | | RELNOTES: None PiperOrigin-RevId: 198765991
* Log warning cleanupGravatar neerajen2018-05-31
| | | | | | | Setting --config in .blazerc that overrode previous options in .blazerc was triggering unnecessary warnings. Fixed by only logging warnings when a user attempts to override an option specified on the command line. RELNOTES: None. PiperOrigin-RevId: 198756160
* Remove redundant @SkylarkModule annotation on JavaInfo.Provider, as its ↵Gravatar cparsons2018-05-31
| | | | | | | skylarkbuildapi interface is annotated. RELNOTES: None. PiperOrigin-RevId: 198743504
* Migrate config-related skylark objects to skylarkbuildapiGravatar cparsons2018-05-31
| | | | | RELNOTES: None. PiperOrigin-RevId: 198735582
* Remove unused codepaths now that non_propagated_deps has been removed as an ↵Gravatar kaipi2018-05-31
| | | | | | attribute. PiperOrigin-RevId: 198735448
* Migrate test-related skylark API to skylarkbuildapiGravatar cparsons2018-05-31
| | | | | RELNOTES: None. PiperOrigin-RevId: 198728350
* Improve DependencyAndroidData VALID_REGEX's robustness.Gravatar jingwen2018-05-31
| | | | | | | | | | | | | | This prevents an invalid flag like `resources:assets:AndroidManifest.xml:::local.bin` to match, as with the case in https://github.com/bazelbuild/bazel/issues/5214 where the R.txt file is missing (shouldn't happen, but let's be defensive) Error message: ``` .. invalid DependencyAndroidData: bazel-out/android-armeabi-v7a-fastbuild/bin/external/androidsdk/com.android.support/_aar/unzipped/resources/support-compat-25.0.0/res:bazel-out/android-armeabi-v7a-fastbuild/bin/external/androidsdk/com.android.support/_aar/unzipped/assets/support-compat-25.0.0/assets:bazel-out/android-armeabi-v7a-fastbuild/bin/external/androidsdk/com.android.support/support-compat-25.0.0_processed_manifest/AndroidManifest.xml:::bazel-out/android-armeabi-v7a-fastbuild/bin/external/androidsdk/com.android.support/support-compat-25.0.0_symbols/local.bin is not in the format 'resources[#resources]:assets[#assets]:manifest:r.txt(:symbols.zip?):symbols.bin' ``` RELNOTES: None. PiperOrigin-RevId: 198724816
* Open up visibility of BuildConfiguration$OutputDirectory for serialization ↵Gravatar janakr2018-05-31
| | | | | | and remove unused repositoryName field from BuildConfiguration. PiperOrigin-RevId: 198723339
* Remove the non_propagated_deps attribute from objc_library and apple_binary. ↵Gravatar kaipi2018-05-30
| | | | | | | This was a workaround created to deal with duplicated symbol errors, but there are now better ways to avoid them than using this attribute. RELNOTES: non_propagated_deps has been removed from objc_library and apple_binary. PiperOrigin-RevId: 198620886
* Remove support for ↵Gravatar cushon2018-05-30
| | | | | | --experimental_one_version_enforcement_use_transitive_jars_for_binary_under_test PiperOrigin-RevId: 198601057
* Fix some tests when run with a JDK9 javabase.Gravatar tomlu2018-05-30
| | | | | RELNOTES: None PiperOrigin-RevId: 198600493
* Special case * in target lookup, work around for #5300.Gravatar twerth2018-05-30
| | | | | | | | * is not an allowed file system character on Windows and checking if it's a directory or file for the wilcard check would throw an InvalidPathException. RELNOTES: None PiperOrigin-RevId: 198595138
* Resolve all super classes even if we have found missing super classes. This isGravatar cnsun2018-05-30
| | | | | | | | | necessary. If we stop resolving the super classes, then the checker will complain that some methods are missign. In fact, these methods exist, but their class is not resolved due to the existing early-break strategy. RELNOTES: none PiperOrigin-RevId: 198594008
* Allow rules that provide resources and assets as deps of android_* rulesGravatar asteinb2018-05-30
| | | | | | | | | | | | | | | | | | | These restrictions were originally put in place to prevent things like android_binary from being used as dependencies of android_library. However, it doesn't actually work - the restrictions are all ORed together, and that means rules that are of the allowed kind OR that have a JavaProvider are allowed. Since all android_binary targets presumably have Java providers, they are presumably all allowed. Slightly expand these restrictions by also allowing rules that have Android resource and asset providers. This will allow us to use Skylark rules to only inherit assets and resources from a target, but not anything else. I don't anticipate this expansion will have unwanted side effects, since virtually all Android targets already have Java providers anyway. RELNOTES: none PiperOrigin-RevId: 198593996
* Modify cc_toolchain_suite rule to allow selection of cc_toolchain label ↵Gravatar rosica2018-05-30
| | | | | | | | | | | | | | | | without reading the CROSSTOOL file. As we want to reroute selecting the toolchain from CROSSTOOL through cc_toolchain, cc_toolchain_suite should have all the necesarry information for obtaining the cc_toolchain label without accessing the CROSSTOOL file. In order to do that, besides the existing <toolchain.targetCpu>|<toolchain.compiler>, we add <cpu> type of key in 'toolchains' attribute of cc_toolchain_suite. Now the selection of cc_toolchain label goes as follows: 1. Return toolchains[<cpu>|<compiler> if it exists 2. Return toolchains[<cpu>] if it exists 3. Fall back to the previous state: return toolchains[<toolchain.targetCpu>|<toolchain.compiler>] RELNOTES: None. PiperOrigin-RevId: 198588849
* Adds support for InlineFileArtifactValue.Gravatar shahan2018-05-30
| | | | PiperOrigin-RevId: 198584000
* Add --apple_enable_auto_dsym_dbg flagGravatar Googler2018-05-30
| | | | | | | | | This new flag can be used to forcibly enable dSYM generation for dbg apple builds, which is especially useful for debugging in remote builds, which may require the use of dSYMs. RELNOTES: Added --apple_enable_auto_dsym_dbg flag. PiperOrigin-RevId: 198577541
* Fix Skylark type typoGravatar asteinb2018-05-30
| | | | | | | | Fortunately or unfortunately, Skylark doesn't seem to validate against this type, so tests don't pick this mistyping up. RELNOTES: none PiperOrigin-RevId: 198568282
* Fix a few tests when run under JDK9.Gravatar tomlu2018-05-30
| | | | | RELNOTES: None PiperOrigin-RevId: 198565423
* Fix Bazel crash on bad config_setting labels.Gravatar gregce2018-05-30
| | | | | | | | | | | | | | | | This used to work more smoothly. But the introduction of ConfiguredTargetAndData adds a Precondition check: https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/skyframe/ConfiguredTargetAndData.java;l=68?q=ConfiguredTargetAndData that crashes if innerConfigurationKey is null. The only way that can happen is when reading a bad label, which Bazel assumes (incorrectly in this case) to be a null-configured source file. Good times. PiperOrigin-RevId: 198557148
* Fix an incorrect example in the platform documentation.Gravatar jcater2018-05-30
| | | | PiperOrigin-RevId: 198556851
* Turn on pseudo locale generation for compilation.Gravatar corysmith2018-05-30
| | | | | RELNOTES: android_binary.aapt_version='aapt2' now supports en_XA and ar_XB PiperOrigin-RevId: 198555305
* Fix `equals()` and `hashCode()` for artifacts: artifacts of different ↵Gravatar Dmitry Lomov2018-05-30
| | | | | | | | | | | | | classes are not equal. Also validate that there are no tree and file artifacts with the same exec path. Fixes #4668. Closes #5284. Change-Id: Id97c0407a476a5bfc697b4ca7b858e3d0c0f8c75 PiperOrigin-RevId: 198540425
* Allow passing the relative symlink resolution policy to the SpawnInputExpanderGravatar ulfjack2018-05-30
| | | | PiperOrigin-RevId: 198535546
* runfiles,cc: C++ runfiles library in @bazel_toolsGravatar Laszlo Csomor2018-05-30
| | | | | | | | | | | | | | Add the C++ runfiles library to @bazel_tools//tools/cpp/runfiles:runfiles. RELNOTES[NEW]: C++,runfiles: to access data-dependencies (runfiles) in C++ programs, use the runfiles library built into Bazel. For usage info, see https://github.com/bazelbuild/bazel/blob/master/tools/cpp/runfiles/runfiles.h Change-Id: I5057a9f477289eea7244c60105e77fc71652a817 Closes #5293. Change-Id: I90cba6fa4c6595c838ae42f9d2c17548c8387e5d PiperOrigin-RevId: 198531849
* Automated rollback of commit 7fcbc8ffdead028d19606fefa2fa3be13781da98.Gravatar laszlocsomor2018-05-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** According to a post-submit regression test, this commit increased Bazel's memory usage slightly beyond some threshold. That event prompted me to consider and question the necessity of this commit. My goal is to make Bazel work on Windows without requiring MSYS, failing the build only if an action is a shell action. Toolchainifying the shell is related to, but not required by this effort. What is required is to fail the build when Bazel tries to create a shell action but the shell is missing, and we have that already with ShToolchain.getPath (which only considers the ShellConfiguration fragment). What's missing for my goal is to reimplement hardcoded shell actions (i.e. SpawnAction.builder().setShellCommand(...)) without using the shell where possible. *** Original change description *** shell toolchain: genrule now uses it genrule() now uses the shell toolchain (as well as the ShellConfiguration config fragment) to look up the shell interpreter's path. Also the CommandHelper no longer looks up the shell interpreter's path itself. Instead its ctor takes the path as an argument. This allows supporting rules that already use the shell toolchain and rules that still only consider the configuration fragment. With these changes genrule() is now able to use the shell interpreter registered as a toolchain, even if there's no `--shell_executable` flag specified (or its value is empty). Subsequent commits will migrate more and more rules to depend on the shell toolchain. This commit takes us closer to: 1. be able to detect the local shell interpreter's location, especially when it's not the default /bin/bash 2. be able to define different shell toolchains (different interpreter paths) for remote builds 3. gracefully fail the build if the machine has no shell installed but the action graph included a shell action See https://github.com/bazelbuild/bazel/issues/4319 Change-Id: I0674c7e2d5917643d87b48b19a1cb43e606ad2f7 Closes #5282. *** RELNOTES: none PiperOrigin-RevId: 198527351
* C++: Hide compilation context from SkylarkGravatar plf2018-05-30
| | | | | RELNOTES:none PiperOrigin-RevId: 198518792
generated by cgit v1.2.3 (git 2.39.1) at 2024-10-18 22:41:44 +0000