aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/test/java/com/google
Commit message (Collapse)AuthorAge
* remote: concurrent blob downloads. Fixes #5215Gravatar buchgr2018-06-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change introduces concurrent downloads of action outputs for remote caching/execution. So far, for an action we would download one output after the other which isn't as bad as it sounds as we would typically run dozens or hundreds of actions in parallel. However, for actions with a lot of outputs or graphs that allow limited parallelism we expect this change to positively impact performance. Note, that with this change the AbstractRemoteActionCache will attempt to always download all outputs concurrently. The actual parallelism is controlled by the underlying network transport. The gRPC transport currently enforces no limits on the concurrent calls, which should be fine given that all calls are multiplexed on a single network connection. The HTTP/1.1 transport also enforces no parallelism by default, but I have added the --remote_max_connections=INT flag which allows to specify an upper bound on the number of network connections to be open concurrently. I have introduced this flag as a defensive mechanism for users who's environment might enforce an upper bound on the number of open connections, as with this change its possible for the number of concurrently open connections to dramatically increase (from NumParallelActions to NumParallelActions * SumParallelActionOutputs). A side effect of this change is that it puts the infrastructure for retries and circuit breaking for the HttpBlobStore in place. RELNOTES: None PiperOrigin-RevId: 199005510
* Make tools in action inputs an error.Gravatar tomlu2018-06-02
| | | | | | | | | | Supporting tools in inputs introduces a slow linear scan. Such tools should be moved to the 'tools' attribute. If --incompatible_no_support_tools_in_action_inputs is set the inputs are scanned, but a helpful error message is issued to the user. Eventually we will remove the slow scanning. Errors will surface in the execution phase instead of during analysis, and the resulting error messages will be less obvious. RELNOTES: None RELNOTES[INC]: With --incompatible_no_support_tools_in_action_inputs enabled, Skylark action inputs are no longer scanned for tools. Move any such inputs to the newly introduced 'tools' attribute. PiperOrigin-RevId: 198996093
* Automated rollback of commit c4e128e2c6d8cacaeba034d6a3195796d50f1745.Gravatar tomlu2018-06-02
| | | | | | | Always generating source jars can lead to action conflicts. RELNOTES: None PiperOrigin-RevId: 198994272
* Removes most ActionInputFileCache functionality.Gravatar shahan2018-06-01
| | | | | | Actual class to be removed in a later change. PiperOrigin-RevId: 198937695
* Allow multiple trimming transition factories to be added.Gravatar mstaib2018-06-01
| | | | | | | | If multiple trimming transition factories are added, they are composed in the order they were added. RELNOTES: None. PiperOrigin-RevId: 198934666
* Amend a skylark test to attempt access of FilesToRunProvider.executableGravatar cparsons2018-06-01
| | | | | | | This serves a regression test of a previous breakage which broke SkylarkCallable method of FilesToRunProvider. RELNOTES: None. PiperOrigin-RevId: 198915920
* Use a new RecursivePackageProvider to ask skyframe for the correct package ↵Gravatar juliexxia2018-06-01
| | | | | | nodes in ConfiguredTargetQueryEnvironemnt. This lets us support recursive target patterns with cquery. PiperOrigin-RevId: 198879650
* Move TARGET_CPU from the CcToolchainProvider to BuildConfiguration.Gravatar jcater2018-06-01
| | | | PiperOrigin-RevId: 198877280
* [java_common.compile] Always generate a source jar.Gravatar elenairina2018-06-01
| | | | | | | | | | | java_common.compile doesn't generate the output source jar when a source jar is the only input for the compilation. This is wrong because the source jar can include APT generated sources. It is also inconsistent with java_library and leads to inconsistent Skylark rules where a declared output will not always have a generating action. RELNOTES: None. PiperOrigin-RevId: 198872718
* Explicitly validate local resources are passed to Java compilation firstGravatar asteinb2018-05-31
| | | | | | | We thought we had a test for this, but it turns out we didn't. Add one now. RELNOTES: none PiperOrigin-RevId: 198772854
* Log warning cleanupGravatar neerajen2018-05-31
| | | | | | | Setting --config in .blazerc that overrode previous options in .blazerc was triggering unnecessary warnings. Fixed by only logging warnings when a user attempts to override an option specified on the command line. RELNOTES: None. PiperOrigin-RevId: 198756160
* Remove unused codepaths now that non_propagated_deps has been removed as an ↵Gravatar kaipi2018-05-31
| | | | | | attribute. PiperOrigin-RevId: 198735448
* Improve DependencyAndroidData VALID_REGEX's robustness.Gravatar jingwen2018-05-31
| | | | | | | | | | | | | | This prevents an invalid flag like `resources:assets:AndroidManifest.xml:::local.bin` to match, as with the case in https://github.com/bazelbuild/bazel/issues/5214 where the R.txt file is missing (shouldn't happen, but let's be defensive) Error message: ``` .. invalid DependencyAndroidData: bazel-out/android-armeabi-v7a-fastbuild/bin/external/androidsdk/com.android.support/_aar/unzipped/resources/support-compat-25.0.0/res:bazel-out/android-armeabi-v7a-fastbuild/bin/external/androidsdk/com.android.support/_aar/unzipped/assets/support-compat-25.0.0/assets:bazel-out/android-armeabi-v7a-fastbuild/bin/external/androidsdk/com.android.support/support-compat-25.0.0_processed_manifest/AndroidManifest.xml:::bazel-out/android-armeabi-v7a-fastbuild/bin/external/androidsdk/com.android.support/support-compat-25.0.0_symbols/local.bin is not in the format 'resources[#resources]:assets[#assets]:manifest:r.txt(:symbols.zip?):symbols.bin' ``` RELNOTES: None. PiperOrigin-RevId: 198724816
* Remove the non_propagated_deps attribute from objc_library and apple_binary. ↵Gravatar kaipi2018-05-30
| | | | | | | This was a workaround created to deal with duplicated symbol errors, but there are now better ways to avoid them than using this attribute. RELNOTES: non_propagated_deps has been removed from objc_library and apple_binary. PiperOrigin-RevId: 198620886
* Modify cc_toolchain_suite rule to allow selection of cc_toolchain label ↵Gravatar rosica2018-05-30
| | | | | | | | | | | | | | | | without reading the CROSSTOOL file. As we want to reroute selecting the toolchain from CROSSTOOL through cc_toolchain, cc_toolchain_suite should have all the necesarry information for obtaining the cc_toolchain label without accessing the CROSSTOOL file. In order to do that, besides the existing <toolchain.targetCpu>|<toolchain.compiler>, we add <cpu> type of key in 'toolchains' attribute of cc_toolchain_suite. Now the selection of cc_toolchain label goes as follows: 1. Return toolchains[<cpu>|<compiler> if it exists 2. Return toolchains[<cpu>] if it exists 3. Fall back to the previous state: return toolchains[<toolchain.targetCpu>|<toolchain.compiler>] RELNOTES: None. PiperOrigin-RevId: 198588849
* Add --apple_enable_auto_dsym_dbg flagGravatar Googler2018-05-30
| | | | | | | | | This new flag can be used to forcibly enable dSYM generation for dbg apple builds, which is especially useful for debugging in remote builds, which may require the use of dSYMs. RELNOTES: Added --apple_enable_auto_dsym_dbg flag. PiperOrigin-RevId: 198577541
* Fix a few tests when run under JDK9.Gravatar tomlu2018-05-30
| | | | | RELNOTES: None PiperOrigin-RevId: 198565423
* Fix Bazel crash on bad config_setting labels.Gravatar gregce2018-05-30
| | | | | | | | | | | | | | | | This used to work more smoothly. But the introduction of ConfiguredTargetAndData adds a Precondition check: https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/skyframe/ConfiguredTargetAndData.java;l=68?q=ConfiguredTargetAndData that crashes if innerConfigurationKey is null. The only way that can happen is when reading a bad label, which Bazel assumes (incorrectly in this case) to be a null-configured source file. Good times. PiperOrigin-RevId: 198557148
* Fix `equals()` and `hashCode()` for artifacts: artifacts of different ↵Gravatar Dmitry Lomov2018-05-30
| | | | | | | | | | | | | classes are not equal. Also validate that there are no tree and file artifacts with the same exec path. Fixes #4668. Closes #5284. Change-Id: Id97c0407a476a5bfc697b4ca7b858e3d0c0f8c75 PiperOrigin-RevId: 198540425
* Allow passing the relative symlink resolution policy to the SpawnInputExpanderGravatar ulfjack2018-05-30
| | | | PiperOrigin-RevId: 198535546
* Automated rollback of commit 7fcbc8ffdead028d19606fefa2fa3be13781da98.Gravatar laszlocsomor2018-05-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** According to a post-submit regression test, this commit increased Bazel's memory usage slightly beyond some threshold. That event prompted me to consider and question the necessity of this commit. My goal is to make Bazel work on Windows without requiring MSYS, failing the build only if an action is a shell action. Toolchainifying the shell is related to, but not required by this effort. What is required is to fail the build when Bazel tries to create a shell action but the shell is missing, and we have that already with ShToolchain.getPath (which only considers the ShellConfiguration fragment). What's missing for my goal is to reimplement hardcoded shell actions (i.e. SpawnAction.builder().setShellCommand(...)) without using the shell where possible. *** Original change description *** shell toolchain: genrule now uses it genrule() now uses the shell toolchain (as well as the ShellConfiguration config fragment) to look up the shell interpreter's path. Also the CommandHelper no longer looks up the shell interpreter's path itself. Instead its ctor takes the path as an argument. This allows supporting rules that already use the shell toolchain and rules that still only consider the configuration fragment. With these changes genrule() is now able to use the shell interpreter registered as a toolchain, even if there's no `--shell_executable` flag specified (or its value is empty). Subsequent commits will migrate more and more rules to depend on the shell toolchain. This commit takes us closer to: 1. be able to detect the local shell interpreter's location, especially when it's not the default /bin/bash 2. be able to define different shell toolchains (different interpreter paths) for remote builds 3. gracefully fail the build if the machine has no shell installed but the action graph included a shell action See https://github.com/bazelbuild/bazel/issues/4319 Change-Id: I0674c7e2d5917643d87b48b19a1cb43e606ad2f7 Closes #5282. *** RELNOTES: none PiperOrigin-RevId: 198527351
* Fix edge case in inferring Java package from path to targetsGravatar asteinb2018-05-29
| | | | | | | | When deriving the Java package from path to a target, actually derive it from the path to the entire target, not just the target's BUILD file. RELNOTES: none PiperOrigin-RevId: 198426047
* Use independently passed assets in aapt2 packaging as wellGravatar asteinb2018-05-29
| | | | | | | | | | | | | | Much earlier, I made a change that allowed passing assets without resources to aapt packaging. Do the same for aapt2 packaging too. The busybox seems to be expecting compiled symbols, so compile assets and pass the compiled version in. (Compiling assets doesn't save any time, but doesn't cost much either, and means that we'll eventually be able to phase out the parsed form entirely. Adapting the Busybox to take parsed assets would probably work too, but getting the code to handle it would be really messy.) RELNOTES: none PiperOrigin-RevId: 198417111
* Change uses of cc_toolchain_type back to plain toolchain_type.Gravatar jcater2018-05-29
| | | | | | This is a prequisite to removing cc_toolchain_type entirely. PiperOrigin-RevId: 198402472
* shell toolchain: genrule now uses itGravatar Laszlo Csomor2018-05-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | genrule() now uses the shell toolchain (as well as the ShellConfiguration config fragment) to look up the shell interpreter's path. Also the CommandHelper no longer looks up the shell interpreter's path itself. Instead its ctor takes the path as an argument. This allows supporting rules that already use the shell toolchain and rules that still only consider the configuration fragment. With these changes genrule() is now able to use the shell interpreter registered as a toolchain, even if there's no `--shell_executable` flag specified (or its value is empty). Subsequent commits will migrate more and more rules to depend on the shell toolchain. This commit takes us closer to: 1. be able to detect the local shell interpreter's location, especially when it's not the default /bin/bash 2. be able to define different shell toolchains (different interpreter paths) for remote builds 3. gracefully fail the build if the machine has no shell installed but the action graph included a shell action See https://github.com/bazelbuild/bazel/issues/4319 Change-Id: I0674c7e2d5917643d87b48b19a1cb43e606ad2f7 Closes #5282. Change-Id: I0674c7e2d5917643d87b48b19a1cb43e606ad2f7 PiperOrigin-RevId: 198394021
* Add a pretty-printer class to SkylarkGravatar Klaus Aehlig2018-05-29
| | | | | | | | | | | With the recording of the results of repository rules (that eventually will lead to an implementation of the WORKSPACE.resolved proposal) bazel started writing out lengthy Skylark values. To make this file more readable for humans, add a Skylkark printer that does at least some basic line breaking and indenting. Change-Id: I469d029421df9212b43747509dd17bd6c64da4a8 PiperOrigin-RevId: 198389112
* Refactor root cause reporting in ConfiguredTargetFunctionGravatar ulfjack2018-05-29
| | | | | | We now track Causes instead of plain Labels, which will allow us to do better reporting in the future. Add basic tests. PiperOrigin-RevId: 198380468
* Remove a reference to an unused field in the CROSSTOOL proto.Gravatar lberki2018-05-29
| | | | | RELNOTES: None. PiperOrigin-RevId: 198378682
* Add CcSysrootAliasRuleGravatar dbabkin2018-05-29
| | | | | RELNOTES:none PiperOrigin-RevId: 198364550
* Skylark: Exposes disabled features from rule context to SkylarkGravatar plf2018-05-28
| | | | | | WANT_LGTM=all RELNOTES:none PiperOrigin-RevId: 198269370
* Fix jdeps handling in java_common.compile(exports)Gravatar cushon2018-05-27
| | | | PiperOrigin-RevId: 198262096
* Start migration of apple skylark build APIGravatar cparsons2018-05-25
| | | | | RELNOTES: None. PiperOrigin-RevId: 198107604
* Skylark: Make 'cfg = "data"' a noop.Gravatar gregce2018-05-25
| | | | | | | This is no longer meaningful with the turndown of (C++) LIPO. PiperOrigin-RevId: 198092974
* support jdeps input in JavaInfo constructorGravatar kmb2018-05-25
| | | | PiperOrigin-RevId: 198081030
* Automated rollback of commit 372fbc2f016157b0331f83a20edad10d4b4cf9f7.Gravatar asteinb2018-05-25
| | | | | | | | | | | | | *** Reason for rollback *** Roll forward with fix: I was assuming that R.txt and symbols files are always set, but they can be null in some cases (especially in the old data processing pipeline). Properly handle them here. RELNOTES: none PiperOrigin-RevId: 198075743
* Remove special casing from processes stat line.Gravatar Googler2018-05-25
| | | | | | | | Fixes #5263 RELNOTES: execution strategies line no longer handles differently the case where all processes have the same strategy. PiperOrigin-RevId: 198057496
* Provide ability to inject custom ConstraintSemantics behavior.Gravatar gregce2018-05-25
| | | | PiperOrigin-RevId: 198053509
* Automated rollback of commit 9d5c323a6e66842cfeb98462cf949dee58710fb8.Gravatar plf2018-05-25
| | | | | | | | *** Reason for rollback *** Crashes lots of targets in nightly blaze-2018.05.24-1: PiperOrigin-RevId: 198049395
* Fix build and run of RealSandboxfsProcessTest.Gravatar jmmv2018-05-25
| | | | | | | | | | | | | | https://github.com/bazelbuild/bazel/commit/56d1b1c3122a3d1ec111baab339631b8c42c2c31 changed the signature of SandboxfsProcess#mount() to take a PathFragment instead of a Path, and this broke the build of the test above. Given that the test is marked as manual, this was not caught by CI. Further, the test was failing to link in the test runner class, so it would not execute properly. The missing dependency was incorrectly removed by unknown commit because it had been specified as a deps instead of a runtime_deps. RELNOTES: None. PiperOrigin-RevId: 198048748
* Extend tests of range()'s list-like behaviorGravatar brandjon2018-05-25
| | | | | | | | | | | | | | This expands on https://github.com/bazelbuild/bazel/commit/33f08e75fa4e7480f8d46b8305ce03a171adefa0 to ensure list-like behavior of range for the following operations - str/repr/type - equality and ordering - append / augmented assignment - concatenation - all of the above with slices This is a prerequisite to both optimizing range() while preserving its current API, as well as changing its API via an incompatible change flag. RELNOTES: None PiperOrigin-RevId: 198046941
* Make @SkylarkModule detection (through superclasses/superinterfaces) ↵Gravatar cparsons2018-05-24
| | | | | | | | | well-defined. Moving forward, if a class not already annotated with @SkylarkModule has SkylarkModule supertypes A and B, then it must be the case that A is a type of B, or B is a type of A. The skylark type of a given class is dictated by the *most specific* superclass annotated with @SkylarkModule. RELNOTES: None. PiperOrigin-RevId: 197946898
* Add test that range() returns a listGravatar brandjon2018-05-24
| | | | | | | This is useful before we optimize range() to return a lazy object that appears like a list to the user. RELNOTES: None PiperOrigin-RevId: 197944773
* Fix SkylarkCallable-annotation detection to appropriately handle methods ↵Gravatar cparsons2018-05-24
| | | | | | | with generic parameters RELNOTES: None. PiperOrigin-RevId: 197932265
* RemoteFileArtifactValue no longer tracks mtime.Gravatar shahan2018-05-24
| | | | PiperOrigin-RevId: 197924560
* Migrate struct() to skylarkbuildapiGravatar cparsons2018-05-24
| | | | | RELNOTES: None. PiperOrigin-RevId: 197915097
* Expose [AndroidAssetsInfo].local_assets, [AndroidAssetsInfo].local_asset_dir ↵Gravatar Googler2018-05-24
| | | | | | | and [AndroidResourcesInfo].r_txt to Skylark. RELNOTES: none PiperOrigin-RevId: 197902129
* Reject files when the first line is indented.Gravatar laurentlb2018-05-24
| | | | | | | | | | | | A bug in the lexer ignored indentation on the first line of a file. This now causes an error. Also, remove the COMMENT token from the lexer. Comments are now accessed separately. This will allow further optimizations in the lexer. It also aligns the code a bit more with the Go implementation. RELNOTES[INC]: Indentation on the first line of a file was previously ignored. This is now fixed. PiperOrigin-RevId: 197889775
* remote/http: simplify download logicGravatar Jakob Buchgraber2018-05-24
| | | | | | | | Netty's HttpClientCodec always emits a LastHttpContent event and so we don't need to track the received bytes manually to know when we are done reading. The HttpBlobStore compares the hashes of the received bytes to give us confidence that what we received is correct. Closes #5244. PiperOrigin-RevId: 197887877
* Delete GlobListGravatar laurentlb2018-05-24
| | | | | RELNOTES: None. PiperOrigin-RevId: 197881012
* Refactor analysis failure reporting, report loading failuresGravatar ulfjack2018-05-24
| | | | | | | | Split the AnalysisFailureEvent into a legacy and a new variant; always post an AnalysisFailureEvent, even if the legacy variant is not posted - this gives us some coverage for loading failures. PiperOrigin-RevId: 197862257
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-08 10:09:06 +0000