aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/tools
Commit message (Collapse)AuthorAge
* [singlejar] Remove redundant test_util depsGravatar Loo Rong Jie2018-08-07
| | | | | | | | | | Side-note: `options_test`, `zip_headers_test` and `zlib_interface_test` now pass on Windows. /cc @laszlocsomor Closes #5781. PiperOrigin-RevId: 207699280
* [singlejar] Set binary flag for WindowsGravatar Loo Rong Jie2018-08-07
| | | | | | | | /cc @laszlocsomor Closes #5779. PiperOrigin-RevId: 207693540
* Automated rollback of commit bab4b04ad09a571615d04aacadf8b3a2b820ba5f.Gravatar corysmith2018-08-06
| | | | | | | | | | | | | *** Reason for rollback *** Appears to cause issues with the new apk. *** Original change description *** Relink instead of convert proto apks. RELNOTES: None PiperOrigin-RevId: 207590807
* Turn on proto based resource shrinking.Gravatar corysmith2018-08-06
| | | | | RELNOTES: None PiperOrigin-RevId: 207584302
* Relink instead of convert proto apks.Gravatar corysmith2018-08-06
| | | | | RELNOTES: None PiperOrigin-RevId: 207575371
* Added functionality to let the ResourceLinker link ProtoApks.Gravatar corysmith2018-08-06
| | | | | RELNOTES: None PiperOrigin-RevId: 207570745
* Allow a ProtoApk to write a manifest as xmlGravatar corysmith2018-08-06
| | | | | | RELNOTES: None PiperOrigin-RevId: 207548204
* Pass digest to Chunker construction when available.Gravatar tomlu2018-08-03
| | | | | | | | | * Refactor Chunker constructor to a builder to reduce constructor overload. * Pass digest into this where we have it * Redo ensureInputsPresent to not lose the missing digests during processing so we can pass them to the Chunker constructor. RELNOTES: None PiperOrigin-RevId: 207297915
* aapt2 creates larger apks when converting from proto.Gravatar corysmith2018-08-03
| | | | | | | Create binary directly then convert to proto format. RELNOTES: None PiperOrigin-RevId: 207273767
* Tool attribute processing added to resource shrinking.Gravatar corysmith2018-08-02
| | | | | RELNOTES: None PiperOrigin-RevId: 207171755
* Expose AndroidResourceUsageAnalyzer methods to enable proto-based resource ↵Gravatar corysmith2018-08-02
| | | | | | | shrinking without tool attributes. RELNOTES: PiperOrigin-RevId: 207132534
* Automatic code cleanup.Gravatar Googler2018-08-01
| | | | PiperOrigin-RevId: 206960449
* Parse raw xml resources for tools annotations.Gravatar corysmith2018-08-01
| | | | | RELNOTES: None PiperOrigin-RevId: 206960066
* Add the ability to do a filtered copy of a ProtoApkGravatar corysmith2018-08-01
| | | | | RELNOTES: None PiperOrigin-RevId: 206949407
* Add support for extracting references and declarations from a proto ↵Gravatar corysmith2018-08-01
| | | | | | | formatted apk RELNOTES: None PiperOrigin-RevId: 206945173
* Automated rollback of commit 0a635c5236ce30ea84b765ce752267992733a649.Gravatar corysmith2018-07-31
| | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Rolling forward with the correct attribute handling. *** Original change description *** Automated rollback of commit 8fe0f45852a620a078013310989396caed273342. *** Reason for rollback *** Breaks a couple of builds due to a bad merge. *** Original change description *** Add apk converted to proto and all attributes from CompiledResources to ResourcesZip. Add new proto format for tool attributes stored in the AndroidDataXml for storing them in the resources.zip. RELNOTES:None PiperOrigin-RevId: 206786645
* Automated rollback of commit 8fe0f45852a620a078013310989396caed273342.Gravatar corysmith2018-07-31
| | | | | | | | | | | | | | *** Reason for rollback *** Breaks a couple of builds due to a bad merge. *** Original change description *** Add apk converted to proto and all attributes from CompiledResources to ResourcesZip. Add new proto format for tool attributes stored in the AndroidDataXml for storing them in the resources.zip. RELNOTES:None PiperOrigin-RevId: 206774364
* Add apk converted to proto and all attributes from CompiledResources to ↵Gravatar corysmith2018-07-31
| | | | | | | | | ResourcesZip. Add new proto format for tool attributes stored in the AndroidDataXml for storing them in the resources.zip. RELNOTES:None PiperOrigin-RevId: 206765679
* Add profiling data to the resources zip.Gravatar corysmith2018-07-31
| | | | | RELNOTES:None PiperOrigin-RevId: 206762523
* Expanding execution log parser to filter by runner type.Gravatar Googler2018-07-30
| | | | | RELNOTES: Execution Log Parser can now, when printing it out, filter the log by runner type PiperOrigin-RevId: 206602183
* [singlejar] cleanup port libraryGravatar Loo Rong Jie2018-07-27
| | | | | | | | /cc @laszlocsomor Closes #5655. PiperOrigin-RevId: 206319221
* Remove Hazelcast dependencyGravatar Philipp Wollermann2018-07-25
| | | | | | | | | | The only remaining use was a testing REST backend in the LRE. I wrote a replacement for that using netty, which we use for our network stuff in Bazel, which means we can now get rid of Hazelcast. :) I'll remove the Hazelcast files in a separate change when this is merged. PiperOrigin-RevId: 205985996
* Windows,tests: port singlejar bash testsGravatar Loo Rong Jie2018-07-23
| | | | | | | | | | Make these tests run on Windows, but currently does not pass. /cc @laszlocsomor Closes #5652. PiperOrigin-RevId: 205658932
* Enable aapt2 for aar_import by adding the AndroidConfiguration fragment.Gravatar corysmith2018-07-18
| | | | | RELNOTES: Fixed compatibility with aar_import when using aapt2. AAPT2 is now supported for Android app builds without resource shrinking. To use it, pass the `--android_aapt=aapt2` flag or define android_binary.aapt_version=aapt2. PiperOrigin-RevId: 205136160
* [singlejar] Change off_t and ssize_t to C standard ptrdiff_tGravatar Loo Rong Jie2018-07-18
| | | | | | | | | | | | | | MSVC does not have `ssize_t` type. MSVC does have `off_t`, but is defined as 32-bit `long` due to legacy reason, this will prevent us from handling large file. Changing `off_t` and `ssize_t` to C standard `ptrdiff_t` for portability and consistency. Changing one instance of `S_ISDIR(st.st_mode)` to `(st.st_mode & S_IFDIR) == S_IFDIR` as MSVC does not have `S_ISDIR` macro. /cc @laszlocsomor Closes #5538. PiperOrigin-RevId: 205038203
* Make default method stubbing resilient to core library renaming, to preserve ↵Gravatar kmb2018-07-17
| | | | | | | | idempotency. RELNOTES: None. PiperOrigin-RevId: 204957400
* [singlejar] Add port library for various MSVC hacksGravatar Loo Rong Jie2018-07-17
| | | | | | | | | | | | A small `port` library with various MSVC hacks. This is needed for future PRs. For non-Windows platforms, `port.cc` will produce an empty object file. #2241 /cc @laszlocsomor Closes #5501. PiperOrigin-RevId: 204889516
* Drop module-info "class" in Android desugaringGravatar kmb2018-07-13
| | | | | | RELNOTES: None. PiperOrigin-RevId: 204492527
* Cleanup Optional usageGravatar corysmith2018-07-11
| | | | | RELNOTES:None PiperOrigin-RevId: 204170706
* [Skylint] Fix a crash on analyzing augmented assignment to IndexExpression.Gravatar Taras Tsugrii2018-07-11
| | | | | | | | | | | | | | | | | | Fixes #5534. Skylint was operating under assumption that all lvalues have a single bound identifier, which is not true for `IndexExpression`s like ``` d["foo"] += "bar" ``` As a result, Skylint would crash. This change makes it handle cases without bound identifiers gracefully. Ideally `IndexExpression` assigments should be analyzed too, but it's a more involved change. Closes #5535. PiperOrigin-RevId: 204109060
* [singlejar] Various portability fixes for MSVCGravatar Loo Rong Jie2018-07-10
| | | | | | | | | | | | | | | - MSVC does not have `errx` functions, so use `diag_errx` etc. instead. - Fix format when trying to print `size_t`, use `%zu` so that the function will handle 32/64-bit `size_t` according to target system automatically. - Adding/guarding a few includes for MSVC. - MSVC does not have `ssize_t`, so replace it with `ptrdiff_t` #2241 /cc @laszlocsomor Closes #5499. PiperOrigin-RevId: 204074420
* Synchronize on process factory to inhibit ETXTBSYGravatar George Gensure2018-07-10
| | | | | | | | | | | | | | | | | | | | | | | | | | Refocus synchronization mechanism to cope with file descriptor set fork- induced races to more tightly constrain concurrent fork/exec pairs. This problem has been observed in bazel proper repeatedly, exhibiting as the iconic ETXTBSY - Text file busy in wide worker pool builds and tests. Evidence that this was discovered by @buchgr is in the comment and change to the embedded ExecutionService implementation, and the description of the race and the need for the synchronization was lifted from that scope to the JavaSubprocessFactory. This factory is a singleton and represents the gateway to all worker process execution, and serves as the correct lock primitive to ensure that file descriptor sets are not duplicated across forks, which gave rise to this issue. To test this, I demonstrated a reproducer presented at https://bugs.java.com/view_bug.do?bug_id=8068370 with 2.4% of invocations in that pathological case exhibiting the issue. With a functionally equivalent change - synchronizing around a processBuilder.start() call - as the only modification to the reproducer, no further failures of any kind were observed, over several hundred runs. Closes #5556. PiperOrigin-RevId: 203947224
* Bazel server, tools: ensure Readers are closedGravatar laszlocsomor2018-07-09
| | | | | | | | | | | | | | | | | | | | Follow-up to commit 59f17d6e0550bf63a0b6ef182e2d63474e058ede. Use try-with-resources to ensure Reader objects are closed eagerly. Eagerly closing Readers avoids hanging on to file handles until the garbage collector finalizes the object, meaning Bazel on Windows (and other processes) can delete or mutate these files. Hopefully this avoids intermittent file deletion errors that sometimes occur on Windows. See https://github.com/bazelbuild/bazel/issues/5512 RELNOTES: none PiperOrigin-RevId: 203771262
* Windows,JNI: more robust nativeCreateJunctionGravatar Laszlo Csomor2018-07-09
| | | | | | | | | | | | | | | | | | | | | | | | Rewrite the CreateJunction function in the Windows JNI library. The new implementation's improvements: - succeeds if the junction already exists with the desired target; hopefully this will fix issue https://github.com/bazelbuild/bazel/issues/5433 - tolerant to concurrent filesystem modifications, e.g. if the junction's path suddenly disappears, the function reports the error correctly Fixes https://github.com/bazelbuild/bazel/issues/5433 Change-Id: I58a2314a00f6edaa7c36c35ba54616168b44eb7d Closes #5528. Change-Id: I9f5dc9237b70a433d0d8c2578a826de3d462d110 PiperOrigin-RevId: 203744515
* windows,launcher: Rlocation function now handles absolute paths correctlyGravatar pcloudy2018-07-06
| | | | | | | Fix https://github.com/bazelbuild/bazel/issues/5516 RELNOTES: None PiperOrigin-RevId: 203448188
* Bazel server: ensure OutputStreams are closedGravatar laszlocsomor2018-07-05
| | | | | | | | | | | | | | | | | | | Use try-with-resources to ensure OutputStreams that we open via FileSystem.OutputStream(path) are closed. Eagerly closing OutputStreams avoids hanging on to file handles until the garbage collector finalizes the OutputStream, meaning Bazel on Windows (and other processes) can delete or mutate these files. Hopefully this avoids intermittent file deletion errors that sometimes occur on Windows. See https://github.com/bazelbuild/bazel/issues/5512 RELNOTES: none PiperOrigin-RevId: 203342889
* [singlejar] Replace setbuffer with setvbufGravatar Loo Rong Jie2018-07-03
| | | | | | | | | | | | `setbuffer` is not available on MSVC, use C-standard `setvbuf` from `stdio.h` instead. `setbuffer` was introduced in https://github.com/bazelbuild/bazel/commit/b4cf5e32a94024c1bfdc6ca432677c31306a3fb5. #2241 /cc @laszlocsomor Closes #5500. PiperOrigin-RevId: 203083781
* Enable android dependency checking in Bazel.Gravatar xingao2018-06-29
| | | | | RELNOTES: None. PiperOrigin-RevId: 202644613
* Add virtual destructor and override to BinaryLauncherBase and subclassesGravatar Loo Rong Jie2018-06-28
| | | | | | | | | | Class with virtual functions must have virtual destructor. Subclasses should annotate overriden functions as `override`. Found by Clang's `-Winconsistent-missing-override`. Closes #5474. PiperOrigin-RevId: 202445230
* runfiles libraries: fix tests and commentsGravatar Laszlo Csomor2018-06-27
| | | | | | | | | | | | | | | | | | | | | | | | | | - Update all runfiles libraries to have the same header comment format, including the build rule to depend on the namespace / header / module to import - Fix runfiles_test.cc to include the right files (recent refactoring commit has split src/main/cpp/util/path.cc from file_[platform].cc) - Change exported variable _rlocation_isabs_pattern in runfiles.bash to be upper-case, so it is visibly a variable and not a function. See https://github.com/bazelbuild/bazel/issues/4460 Change-Id: I17e18308506ab9f5c9f410ef6bc6b9df912d42a9 Closes #5481. Change-Id: I17e18308506ab9f5c9f410ef6bc6b9df912d42a9 PiperOrigin-RevId: 202291629
* Replace //tools/defaults:jdk to //tools/jdk:*Gravatar dbabkin2018-06-26
| | | | | | | As //tools/defaults will be deprecated soon. All usages of //tools/defaults:jdk and //tools/defaults:java_toolchain should be replaced by corresponding targets in //tools/jdk/BUILD package RELNOTES:none PiperOrigin-RevId: 202114489
* Windows: Native launcher now works with unicode.Gravatar Yun Peng2018-06-25
| | | | | | | | | | | | | | The native launcher can now launch Java and Bash binary in directory with non-English characters. Unfortunately, python doesn't support running python zip file under directory with non-English characters. eg. python ./??/bin.zip will still fail. See https://github.com/bazelbuild/bazel/issues/4473 Change-Id: I77fe9cdaabffc2e0d25c7097da5c0c9333a9c4a3 PiperOrigin-RevId: 201939391
* Move HashFunction out of FileSystem, and turn it into a class, instead of an ↵Gravatar ccalvarin2018-06-21
| | | | | | | | | enum. Now that we aren't using enum names for the hash functions, we also accept the standard names, such as SHA-256. RELNOTES: None. PiperOrigin-RevId: 201624286
* Fail when resources use invalid java identifiers.Gravatar corysmith2018-06-15
| | | | | RELNOTES:None PiperOrigin-RevId: 200766836
* Adding a tool to parse the execution logs.Gravatar olaola2018-06-15
| | | | | | | | For now, the tool simply displays the log as text. TESTED=ran it RELNOTES: A tool to parse the Bazel execution log. PiperOrigin-RevId: 200718299
* Remove usage of COMPILER_MSVC in Bazel and ijarGravatar Loo Rong Jie2018-06-11
| | | | | | | | | | | | | Convert most `COMPILER_MSVC` to `_WIN32` (as they apply to Windows platform, not MSVC compiler). Only `src/tools/singlejar/zip_headers.h` and `src/main/cpp/util/md5.h` actually need `_MSC_VER`. `COMPILER_MSVC` in `third_party/protobuf` are not removed. They can be fixed by updating dependency to newer version. /cc @meteorcloudy Closes #5350. Change-Id: Ibc131abfaf34a0cb2bd338549983ea9d28eaabfe PiperOrigin-RevId: 200019793
* [launcher] Don't pass std::string to C varargs functionGravatar Loo Rong Jie2018-06-07
| | | | | | | | | | | | | `die` passes parameters to `vfprintf`. Passing `std::string` to this function might crash the program. Found by Clang on Windows. /cc @meteorcloudy Closes #5339. Change-Id: Iff7aa766770771262f91990774ff4d8de1fbeb5c PiperOrigin-RevId: 199630717
* Move path-manipulation functions to own library file.Gravatar ccalvarin2018-06-05
| | | | | | | | | | | Leave functions that make file accesses in the file library, and general blaze utilities in the blaze_util file, but move the functions that boil down to string manipulation and path formatting to their own file. (With the exception of getCWD, since absolute path syntax is relevant here.) Doing this largely to consolidate all Windows path control into a single place, so that it's easier to notice inconsistencies. For instance, ConvertPath currently makes Windows paths absolute, but not Posix paths, and MakeAbsolute relies on this behavior. In addition, JoinPath assumes Posix path syntax, which leads to some odd looking paths. These will be fixed in a followup change. (Found these issues while working on #4502, trying to fix the windows-specific system bazelrc.) RELNOTES: None. PiperOrigin-RevId: 199368226
* remote: concurrent blob downloads. Fixes #5215Gravatar buchgr2018-06-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change introduces concurrent downloads of action outputs for remote caching/execution. So far, for an action we would download one output after the other which isn't as bad as it sounds as we would typically run dozens or hundreds of actions in parallel. However, for actions with a lot of outputs or graphs that allow limited parallelism we expect this change to positively impact performance. Note, that with this change the AbstractRemoteActionCache will attempt to always download all outputs concurrently. The actual parallelism is controlled by the underlying network transport. The gRPC transport currently enforces no limits on the concurrent calls, which should be fine given that all calls are multiplexed on a single network connection. The HTTP/1.1 transport also enforces no parallelism by default, but I have added the --remote_max_connections=INT flag which allows to specify an upper bound on the number of network connections to be open concurrently. I have introduced this flag as a defensive mechanism for users who's environment might enforce an upper bound on the number of open connections, as with this change its possible for the number of concurrently open connections to dramatically increase (from NumParallelActions to NumParallelActions * SumParallelActionOutputs). A side effect of this change is that it puts the infrastructure for retries and circuit breaking for the HttpBlobStore in place. RELNOTES: None PiperOrigin-RevId: 199005510
* Format all bzl files with buildifierGravatar vladmos2018-06-01
| | | | | | This CL aslo enables a presubmit check for correct formatting of all bzl files in //third_party/bazel. PiperOrigin-RevId: 198857055
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-05 18:19:05 +0000