aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/remote/Chunker.java
Commit message (Collapse)AuthorAge
* Refactor the FileSystem API to allow for different hash functions.Gravatar buchgr2017-11-30
| | | | | | | | | | | | | Refactor the FileSystem class to include the hash function as an instance field. This allows us to have a different hash function per FileSystem and removes technical debt, as currently that's somewhat accomplished by a horrible hack that has a static method to set the hash function for all FileSystem instances. The FileSystem's default hash function remains MD5. RELNOTES: None PiperOrigin-RevId: 177479772
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* ActionInputFileCache: move getMetadata to a new super-interfaceGravatar ulfjack2017-09-11
| | | | | | Update the callers that only need getMetadata to use the new interface. PiperOrigin-RevId: 167992239
* remote: Chunker.reset() should release resources.Gravatar buchgr2017-07-17
| | | | | RELNOTES: None. PiperOrigin-RevId: 161970540
* remote: Chunker should open files lazily.Gravatar buchgr2017-07-14
| | | | | | | | | | The Chunker should only open a file on the first call to next(). We noticed that when remote executing with hundreds of actions in parallel bazel would sometimes run out of file descriptors. That's because on creating a new Chunker object it would already open a file, eventhough the first call to next would happen at a much later stage. PiperOrigin-RevId: 161923568
* remote: Rewrite ChunkerGravatar buchgr2017-07-12
| | | | | | | | | | | | | | | | | - Remove the Chunker.Builder and use the Chunker constructors instead. - Fix a correctness bug, where the chunker would ignore the return value of InputStream.read(byte[]). - Have Chunk.getData() return a ByteString as opposed to a byte[]. All callsides need ByteString objects and this change makes the subsequent change possible. - Have the Chunker use a preallocated byte[] in order to avoid allocating a new one on every call to next(). RELNOTES: None. PiperOrigin-RevId: 161637158
* remote: Add a SingleSourceBuilder to the Chunker.Gravatar buchgr2017-07-07
| | | | | | | | Prepare the Chunker for an upcoming refactoring, where it will no longer support multiple input sources and digest filtering. RELNOTES: None. PiperOrigin-RevId: 161189759
* Slight refactoring, functional noop: uploadChunks will need to get a ↵Gravatar olaola2017-06-30
| | | | | | | Chunker.Builder in my next change, so all the from factory methods are removed. TESTED=unit test PiperOrigin-RevId: 160594730
* Cleanup: addressing comments from commit ↵Gravatar olaola2017-06-16
| | | | | | | 9eea05d068a06ab642dd9d86d46ee5fa2e36b02e. RELNOTES: n/a PiperOrigin-RevId: 158988688
* Switching Bazel to use the new remote execution API: ↵Gravatar olaola2017-06-09
| | | | | | | | | https://docs.google.com/document/d/1AaGk7fOPByEvpAbqeXIyE8HX_A3_axxNnvroblTZ_6s/edit Also refactored away the various *Interface* files, no need since unit testing can be done with mocking the appropriate gRPC Impl classes directly (see tests). This also fixes the RemoteSpawnRunner, which should use different objects for remote caching and remote execution, the same way RemoteSpawnStrategy does. RELNOTES: n/a PiperOrigin-RevId: 158473700
* Handle null action inputs in TreeNodeRepositoryGravatar ulfjack2017-05-03
| | | | | | | | The SpawnInputExpander can return null action inputs to indicate that we should create an empty file at the corresponding location, without a corresponding input file. PiperOrigin-RevId: 154832564
* Refactoring the chunking logic out of GrpcActionCache, and cleaning it up. AlsoGravatar olaola2017-04-06
| | | | | | | | | | | | added tests. This should not conflict with any of Ulf's upcoming CLs. I need it in preparation for the next CL, which fixes the bug that I don't limit the message size according to grpcChunkSizeBytes and don't do chunking in RemoteWorker.java, causing it to fail on uploading large outputs. TESTED=unit / integration tests RELNOTES: n/a PiperOrigin-RevId: 152385956
* Automated g4 rollback of commit 520f947580634cb9b62020fba4c93d5e78044ccf.Gravatar mstaib2017-04-06
| | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Broke mac_jdk7_test. *** Original change description *** Refactoring the chunking logic out of GrpcActionCache, and cleaning it up. Also added tests. This should not conflict with any of Ulf's upcoming CLs. I need it in preparation for the next CL, which fixes the bug that I don't limit the message size according to grpcChunkSizeBytes and don't do chunking in RemoteWorker.java, causing it to fail on uploading large outputs. TESTED=unit / integration tests RELNOTES: n/a PiperOrigin-RevId: 152270056
* Refactoring the chunking logic out of GrpcActionCache, and cleaning it up. AlsoGravatar olaola2017-04-06
added tests. This should not conflict with any of Ulf's upcoming CLs. I need it in preparation for the next CL, which fixes the bug that I don't limit the message size according to grpcChunkSizeBytes and don't do chunking in RemoteWorker.java, causing it to fail on uploading large outputs. TESTED=unit / integration tests RELNOTES: n/a PiperOrigin-RevId: 152258232
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-18 09:26:53 +0000