aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/tools
Commit message (Collapse)AuthorAge
* Remove an unused constructorGravatar cushon2017-07-07
| | | | PiperOrigin-RevId: 161125326
* Rewrite the watcher / execution servers to use a thread-pool for executionGravatar ulfjack2017-07-06
| | | | | | | Also change them to use ListenableFuture, so multiple watchers can watch the same action execution. PiperOrigin-RevId: 160989277
* Rewrite blob upload to use temporary filesGravatar ulfjack2017-07-06
| | | | | | | | | | | | | | Previously, it was allocating in-memory buffers for upload, which caused it to run out of memory on large file uploads (or on many small uploads running simultaneously). Unfortunately, we don't create the temporary file in the right location (due to separation of the BlobStore from the ByteStreamServer), so we copy the file again to write it to the OnDiskBlobStore, which isn't ideal. There'll need to be another BlobStore API change to make that work, when the InMemoryBlobStore is gone. PiperOrigin-RevId: 160974550
* fix singlejar signedness problemsGravatar Benjamin Peterson2017-07-05
| | | | | | | | | | | | while (total_written < count) { ~~~~~~~~~~~~~~^~~~~~~ Most of these can be fixed by simply by storing the return value of various functions in the right variable type. E.g., cen_offset32 can naturally be stored in uin32_t rather than uint64_t. Change-Id: I9f188ca541e4fe2587c636670807f56887e1655e PiperOrigin-RevId: 160897953
* Add an on-disk storage option for the remote workerGravatar ulfjack2017-07-05
| | | | | | | | | | I'm planning to switch the remote worker over to the on-disk storage system, and delete the in-memory option. Currently, running a Bazel build with the in-memory storage system uses up over 50% of the total available memory on my machine (I only have 32 GB total), and grinds it to a complete halt (unless I close most of my apps) on a simple build of //src:bazel. PiperOrigin-RevId: 160877546
* Move the SimpleBlobStore and implementations to a subpackageGravatar ulfjack2017-07-05
| | | | | | | | | | Also extend the API to throw exceptions rather than having to wrap or swallow. This is in preparation for adding yet another implementation using an on-disk cache. Having the RemoteWorker keep everything in memory is not good for my sanity. PiperOrigin-RevId: 160871586
* Identify which methods are used in invokedynamic, and only desugar theseGravatar cnsun2017-07-03
| | | | | | | methods. RELNOTES: None PiperOrigin-RevId: 160663025
* Windows, Android: ScopedTemporaryDirectory bugfixGravatar Laszlo Csomor2017-07-03
| | | | | | | | | | | | ScopedTemporaryDirectory now makes all files writable before attempting to delete them. This is important on Windows where readonly files cannot be deleted, the attempt resulting in an exception. See https://github.com/bazelbuild/bazel/issues/3264 Change-Id: If79478a4b419c05d77ce89cc30cb701d42df1b75 PiperOrigin-RevId: 160644599
* Enforce the new category and effect tags.Gravatar ccalvarin2017-07-03
| | | | | | | | | | All options need to explicitly list their category and effect. If they are uncategorized, this makes the lack of information obvious. Remove defaults from the annotation to enforce this. Also enforce the sanity check that no option should have UNKNOWN or NO_OP effects listed with other effect tags. Includes some last default sets for options I missed in the previous mass-setting change, and some that were added since. PiperOrigin-RevId: 160641861
* Implement retry logic for the gRPC calls in remote execution and caching. TheGravatar olaola2017-06-30
| | | | | | | | retry strategy may need tuning. Other behavior changes: swallowing gRPC CANCELLED errors when the thread is interrupted, as these are expected and just make debugging difficult. Also, distinguishing between the gRPC DEADLINE_EXCEEDED caused by the actual command timing out on the server vs. other causes (the former should not be retriable, while the latter should retry). TESTED=unit tests, remote worker on Bazel PiperOrigin-RevId: 160605830
* Slight refactoring, functional noop: uploadChunks will need to get a ↵Gravatar olaola2017-06-30
| | | | | | | Chunker.Builder in my next change, so all the from factory methods are removed. TESTED=unit test PiperOrigin-RevId: 160594730
* Add the rest of the categories transition values to android options.Gravatar ccalvarin2017-06-30
| | | | | | Automated formatting fixes standardize the @Option annotation. PiperOrigin-RevId: 160582653
* Flip the flag --rewrite_calls_to_long_compare to false by default. NowGravatar cnsun2017-06-30
| | | | | | | | desugaring Long.compare(long, long) is only enabled if either --min_sdk_version < 19 or --rewrite_calls_to_long_compare is specified. RELNOTES: set --rewrite_calls_to_long_compare to false by default. PiperOrigin-RevId: 160578028
* Add categories transition values to android options.Gravatar ccalvarin2017-06-30
| | | | | | Automated formatting fixes standardize the @Option annotation. PiperOrigin-RevId: 160567787
* Open source dexer tests.Gravatar ajmichael2017-06-29
| | | | | RELNOTES: None PiperOrigin-RevId: 160461708
* When we access a field of an interface, if the field is of type long or double,Gravatar cnsun2017-06-29
| | | | | | | then we use pop2 to pop the value out of the stack. RELNOTES: None PiperOrigin-RevId: 160453904
* Fix extra bytes and missing resources for roboelectric tests:Gravatar Googler2017-06-29
| | | | | | | | | *) Changed the GenerateRobolectricResourceSymbolsAction to merge duplicate namespaces from the dependency list input. *) RClassGenerator throws if an existing R.class file exists for that package. *) New test for duplicate package dependencies RELNOTES: none PiperOrigin-RevId: 160436937
* Access interface constants to explicitly trigger the execution of interfaceGravatar cnsun2017-06-29
| | | | | | | | | | | | | | | | | | | | | initializers. The problem is that when we desugar default methods, the static intializer of interface will not be executed. The JVM spec says that if an interface has default methods, then when it is loaded, it will also be initialized. After we desugar such an interface, its default methods are removed, and when we load the interface, the <clinit> will not be executed. This CL checks whether an interface has default methods and fields. If yes (needs to be initialized when the interface is loaded), it injects field access code to access the interface fields in the <clinit> of the companion class. We also create a constant $$CONSTANT$$ in the companion class. Then for all the classes that implement the interface, we inject GETSTATIC in the class <clinit> to the $$CONSTANT$$ of the companion class of the interface. This indirection is to avoid the IllegalAccessError when the interface is package private. Note that accessing an arbitrary interface field does not guarantee the interface will be initialized. We need to access the field that is initialized in the interface static initializer. RELNOTES: None PiperOrigin-RevId: 160414671
* Add the default category and tag to all options.Gravatar ccalvarin2017-06-28
| | | | | | | | Move the default from the annotation to every mention. This makes the incompleteness explicit. Will add the defaults to test targets in a separate change. Once all dependencies are cleaned up, the Option annotation will no longer allow options without the documentationCategory or effectTag, to prevent new options being added without categories while we migrate to the new option categorization. PiperOrigin-RevId: 160281252
* Fixes #3188: Implement sandboxing for remote_worker on Linux.Gravatar Philipp Wollermann2017-06-28
| | | | | | | RELNOTES: Bazel's remote_worker backend for remote execution supports sandboxing on Linux now. Check https://github.com/bazelbuild/bazel/blob/master/src/tools/remote_worker/README.md for details. Change-Id: I918b0291472c8c7d4884850d9ca0f03674ef2f31 PiperOrigin-RevId: 160266742
* Simplify the RemoteActionCache interfaceGravatar ulfjack2017-06-27
| | | | | | | | | | | | | | | | | | | - merge all the inputs upload functionality into a single ensureInputsPresent method - merge all of the action result upload functionality into a single upload method - merge all the download functionality into a single download method This significantly simplifies the caller of this interface, and opens the door for additional performance improvements in implementations which now have more control over the upload / download flows; in particular, in the gRPC case, we can upload stdout / stderr using the existing chunker - upload of stdout / stderr is no longer serialized with file upload. In particular, the CachedLocalSpawnRunner test becomes much simpler, since it no longer needs to handle the previous more complex upload code path. PiperOrigin-RevId: 160260161
* Make it an error to pass --multidex=off with --main-dex-list orGravatar ajmichael2017-06-26
| | | | | | | | | | | --minimal-main-dex. This matches dx's behavior. Previously, DexFileMerger would override --multidex=off with --multidex=minimal if either --main-dex-list or --minimal-main-dex were passed. RELNOTES: Incremental dexing errors on combination of --multidex=off and either --main-dex-list or --minimal-main-dex. PiperOrigin-RevId: 159858242
* Refactor our RemoteWorker for easier maintenance in the future.Gravatar Philipp Wollermann2017-06-22
| | | | | | | | | | This contains no real functional changes apart from slightly tweaked log messages. RELNOTES: None. Change-Id: I84fb5cf7b1e31a5ec693bbf39e22573e1750ff04 PiperOrigin-RevId: 159700159
* Make source code of singlejar and ijar available from @bazel_tools. ,Gravatar Xin Gao2017-06-22
| | | | | | | | | | Please refer to patch set 9 and its CI run for usage and test results. In practice, users should create their own java_toolchain rule in their project's BUILD file, and set the two attributes like above instead of modifying //tools/jdk/BUILD. Change-Id: Ic880f243086b00a58d453a8139ba4c957fe54bc7 PiperOrigin-RevId: 159694649
* Automated rollback of commit 317e0de2113f0361c9dbf98653ceda4efea3f3e2.Gravatar Googler2017-06-20
| | | | | | | | | | | | | | | | | | | | | *** Original change description *** Access interface constants to explicitly trigger the execution of interface initializers. The problem is that when we desugar default methods, the static intializer of interface will not be executed. The JVM spec says that if an interface has default methods, then when it is loaded, it will also be initialized. After we desugar such an interface, its default methods are removed, and when we load the interface, the <clinit> will not be executed. This CL checks whether an interface has default me... *** ROLLBACK_OF=159496992 NO_SQ=PURE_ROLLBACK RELNOTES: None PiperOrigin-RevId: 159545752
* Access interface constants to explicitly trigger the execution of interfaceGravatar cnsun2017-06-20
| | | | | | | | | | | | | | | | | | | | | initializers. The problem is that when we desugar default methods, the static intializer of interface will not be executed. The JVM spec says that if an interface has default methods, then when it is loaded, it will also be initialized. After we desugar such an interface, its default methods are removed, and when we load the interface, the <clinit> will not be executed. This CL checks whether an interface has default methods and fields. If yes (needs to be initialized when the interface is loaded), it injects field access code to access the interface fields in the <clinit> of the companion class. We also create a constant $$CONSTANT$$ in the companion class. Then for all the classes that implement the interface, we inject GETSTATIC in the class <clinit> to the $$CONSTANT$$ of the companion class of the interface. This indirection is to avoid the IllegalAccessError when the interface is package private. Note that accessing an arbitrary interface field does not guarantee the interface will be initialized. We need to access the field that is initialized in the interface static initializer. RELNOTES: None PiperOrigin-RevId: 159496992
* Adding support for SHA256 for remote execution. Switch remote execution to useGravatar olaola2017-06-20
| | | | | | | | | the currently defined hash function for blobs. Some refactoring. Adding an option to set the hash function in the remote worker, defaulting to the current behavior (unfortunately it is a build option, have not found a clean way to specify it at runtime). BUG=62622420 TESTED=remote worker RELNOTES: none PiperOrigin-RevId: 159473116
* Roll forward R generation for Robolectric behind a flag to enable testingGravatar Googler2017-06-19
| | | | | RELNOTES: none PiperOrigin-RevId: 159263527
* Automated g4 rollback of commit 5c2dc673a3ad2aa52f25c6f7d767671ba3e9b02c.Gravatar corysmith2017-06-16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Appears that the generated R classes are not correct, causing test failures. *** Original change description *** Automated g4 rollback of commit 1be84ee92a825694b0c0026ac77134f8a084bd9c. *** Reason for rollback *** Rolling forward with the generated resources directory being created for tests without resources. *** Original change description *** Automated g4 rollback of commit d028d7854d3e95d97143945a1ec32944e5e4594b. *** Reason for rollback *** Breaks 1000+ targets in the depot: [] *** Original change description *** Generate pre-reconciled ids R classes for dependen... *** RELNOTES: None PiperOrigin-RevId: 159032530
* Switching to Watcher API instead of wait_for_completion, in preparation forGravatar olaola2017-06-14
| | | | | | | | | deprecating the wait_for_completion field. Note on errors: in the RemoteWorker, I currently handle all errors as onError of the watch call. Other options are: pass them as the operation error field, and pass some of them as the onError of the execute call. For now, I'm just using the simplest option; the Bazel client is ready to handle all possible options. RELNOTES: none PiperOrigin-RevId: 158974207
* Automated g4 rollback of commit 1be84ee92a825694b0c0026ac77134f8a084bd9c.Gravatar corysmith2017-06-13
| | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Rolling forward with the generated resources directory being created for tests without resources. *** Original change description *** Automated g4 rollback of commit d028d7854d3e95d97143945a1ec32944e5e4594b. *** Reason for rollback *** Breaks 1000+ targets in the depot: [] *** Original change description *** Generate pre-reconciled ids R classes for dependency libraries with resources in the transitive closure of robolectric tests. RELNOTES: None PiperOrigin-RevId: 158756991
* Extract FieldInfo as a top-level class. In the following CL, I will use thisGravatar cnsun2017-06-12
| | | | | | | class in DefaultMethodClassFixer to fix the bug b/38255926 RELNOTES: n/a PiperOrigin-RevId: 158561127
* Make singlejar aware of FreeBSDGravatar Klaus Aehlig2017-06-09
| | | | | | | | | | | | | The C implementation of singlejar verifies that it is built for a supported system; currently those are 64-bit linux and apple systems. However, the code is portable enough that it can easily be adapted to run on 64-bit FreeBSD systems as well. In fact, only the path of a single header file is different, and we need to link with -lm. Fixes #3096. Change-Id: Id69542ad940f1161aceb55eb8f5dabda2f486747 PiperOrigin-RevId: 158533955
* Switching Bazel to use the new remote execution API: ↵Gravatar olaola2017-06-09
| | | | | | | | | https://docs.google.com/document/d/1AaGk7fOPByEvpAbqeXIyE8HX_A3_axxNnvroblTZ_6s/edit Also refactored away the various *Interface* files, no need since unit testing can be done with mocking the appropriate gRPC Impl classes directly (see tests). This also fixes the RemoteSpawnRunner, which should use different objects for remote caching and remote execution, the same way RemoteSpawnStrategy does. RELNOTES: n/a PiperOrigin-RevId: 158473700
* Erase <clinit> of interfaces to prevent <clinit> from execution during desugar.Gravatar cnsun2017-06-08
| | | | | | | For all declared non-premitive fields, this CL sets each one to null. RELNOTES: n/na PiperOrigin-RevId: 158305638
* Move bridge methods for default methods down to the implementing classes.Gravatar cnsun2017-06-07
| | | | | RELNOTES: n/na PiperOrigin-RevId: 158162658
* Automated g4 rollback of commit d028d7854d3e95d97143945a1ec32944e5e4594b.Gravatar lpino2017-06-07
| | | | | | | | | | | | | | *** Reason for rollback *** Breaks 1000+ targets in the depot: [] *** Original change description *** Generate pre-reconciled ids R classes for dependency libraries with resources in the transitive closure of robolectric tests. RELNOTES: None PiperOrigin-RevId: 158159550
* protobuf: Use bazel's native java_proto_library rules.Gravatar Jakob Buchgraber2017-06-06
| | | | | | | | | Additional changes: - Introduce a Skylark macro java_library_srcs that provides the source jars of a java_*_library rule. - Remove bazel's own java_proto_library implementation. Change-Id: I18f2259bc75ca0fb32dcd8a6a857c609bd2c7773 PiperOrigin-RevId: 158146210
* Improve the error message when Desugar fails to load a class from theGravatar cnsun2017-06-05
| | | | | | | classpath. Now it prints the name of the class that is under desugaring. RELNOTES: n/a PiperOrigin-RevId: 157889131
* Generate pre-reconciled ids R classes for dependency libraries with resourcesGravatar corysmith2017-06-02
| | | | | | | in the transitive closure of robolectric tests. RELNOTES: None PiperOrigin-RevId: 157762190
* Simplify the remote workerGravatar ulfjack2017-05-31
| | | | | | | | | | | | - Reduce scope of try-catch blocks - Handle non-zero exit the same as zero exit - Basic infrastructure to handle time outs, currently hard-coded to 15 mins Some of this code is copied from LocalSpawnRunner. Ideally, we'd reuse that implementation instead of writing yet another one, but that will have to wait for some more refactoring. PiperOrigin-RevId: 157506934
* Enable desugaring try-with-resources by default.Gravatar cnsun2017-05-31
| | | | | RELNOTES: none PiperOrigin-RevId: 157490771
* Remote tests should not depend on lib:runtimeGravatar buchgr2017-05-31
| | | | | | | | Move AuthAndTLSOptions to its own package, so that tests/remote no longer depends on lib:runtime. RELNOTES: None. PiperOrigin-RevId: 157469629
* Migrate Java tests to Truth.Gravatar lberki2017-05-30
| | | | | | RELNOTES: None. PiperOrigin-RevId: 157446717
* Remote worker: skip non-existent files after action executionGravatar ulfjack2017-05-30
| | | | | | | | | | | | | | | | | Tests basically always finish with non-existent files, and this is not an error on the server side. Instead, the client (Bazel) checks whether all the files it expects are present. The test didn't catch this because it was silently falling back to local execution. Non-existent files were leading to an IOException, which caused the remote worker to log nothing, and silently return an error with no output. Log errors in the remote worker to make future debugging easier. Fixes #2887. PiperOrigin-RevId: 157400131
* For static interface method, when we move it to the companion class,Gravatar cnsun2017-05-26
| | | | | | | | append a suffix to the method name, so as to avoid name clash in the companion class. RELNOTES: n/a PiperOrigin-RevId: 157021961
* Fix bug with combining resource attributes. Attributes were reporting that ↵Gravatar apell2017-05-26
| | | | | | | they were over-writable in all cases instead of differentiating by attribute type. RELNOTES: None. PiperOrigin-RevId: 156999284
* Add a new action for generating reconciled R classes for Robolectric.Gravatar corysmith2017-05-23
| | | | | | | | | This includes some refactoring: * Move the symbol deserialization our of the merger and into the ParsedAndroidData (probably move again.) * Change the FailedFutureAggregator generics to work more callables RELNOTES: None PiperOrigin-RevId: 156863698
* Make Desugar work for relative dumpDirectory paths.Gravatar Googler2017-05-23
| | | | | | | | | | | | | Before CL 155913466, Desugar was not working when dumpDirectory was a relative path because of an extra/incorrect resolve() that ended up duplicating such a relative path. CL 155913466 fixed this by converting Desugar's dumpDirectory to an absolute path. However, CL 156257767 introduced a check that Desugar's dumpDirectory must match the one in Java's InnerClassLambdaMetafactory, which broke the case of a relative path again. Therefore, this CL undoes the conversion to absolute path again (so that Desugar's path matches that in InnerClassLambdaMetafactory). Instead, to make relative paths work, it drops the unnecessary/incorrect call to resolve(). A regression test is left for a future CL (I don't have a dev environment set up that would allow me to easily write one). RELNOTES: None. PiperOrigin-RevId: 156852882
* Make checking of lambda dump directory accept relative path.Gravatar cnsun2017-05-23
| | | | | | | | | | | | | | Before, we use Path.equals(Object), which will fail if we use relative path in the system property. Now I use Files.isSameFile which will check two paths point to the same file system resource. In https://github.com/bazelbuild/bazel/commit/69e855c7b0f0f7899a69a882cba0abd304233c97, I introduce the check to test whether setting the system property for lambda meta factory succeeds. However, the check does not consider relative path, which will always fail. RELNOTES: n/a PiperOrigin-RevId: 156796457
generated by cgit v1.2.3 (git 2.39.1) at 2024-10-23 21:28:46 +0000