aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/vfs/JavaIoFileSystem.java
Commit message (Collapse)AuthorAge
* Automated rollback of commit 7e87730de985b7099b9b683571d58efdaab70890.Gravatar ccalvarin2018-07-27
| | | | | | | | | | | | | | | | | *** Reason for rollback *** Go back to the default constructor - instead of requiring everywhere to know the correct hash function, we'll have the default rely on global state. It will make transition easier, even if it makes the origin of the hash less obvious. *** Original change description *** Remove default MD5 in most of Bazel's virtual filesystems. This forces the ex-default to be explicit in a lot of tests, but I'd rather that than have the risk of implicit md5-use in production code. To keep this CL smaller, do not remove the default from UnixFS quite yet. RELNOTES: None. PiperOrigin-RevId: 206358838
* Remove default MD5 in most of Bazel's virtual filesystems.Gravatar ccalvarin2018-07-26
| | | | | | | | | This forces the ex-default to be explicit in a lot of tests, but I'd rather that than have the risk of implicit md5-use in production code. To keep this CL smaller, do not remove the default from UnixFS quite yet. RELNOTES: None. PiperOrigin-RevId: 206223521
* Remove redundancy in DigestHashFunction use in FileSystem.Gravatar ccalvarin2018-07-23
| | | | | | | Each FileSystem instance has a digest function, but the getDigest and getHashDigest functions also accepted their own custom parameter functions. We only support 1 hash per filesystem instance, these parameters are redundant. RELNOTES: None. PiperOrigin-RevId: 205758571
* vfs: fix race condition in JavaIoFileSystem.deleteGravatar Laszlo Csomor2018-07-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | JavaIoFileSystem.delete() now uses Files.deleteIfExists() to delete files and empty directories. The old code used to have a race condition: a file could have been deleted or changed to a non-directory between checking it's a directory and listing its contents. WindowsFileSystem.delete() now uses the DeletePath JNI method (wrapped by WindowsFileOperations.deletePath) which is more robust than JavaIoFileSystem.delete(), because it can delete readonly files and can tolerate some concurrent filesystem modifications. Fixes https://github.com/bazelbuild/bazel/issues/5513 Change-Id: I5d2720afff8dfd3725880a6d7408d22de5935c3d Closes #5527. Change-Id: I5d2720afff8dfd3725880a6d7408d22de5935c3d PiperOrigin-RevId: 203720575
* Move HashFunction out of FileSystem, and turn it into a class, instead of an ↵Gravatar ccalvarin2018-06-21
| | | | | | | | | enum. Now that we aren't using enum names for the hash functions, we also accept the standard names, such as SHA-256. RELNOTES: None. PiperOrigin-RevId: 201624286
* Change profiling to only accept strings for its "description" argument. ↵Gravatar janakr2018-04-01
| | | | | | Profiling can hold onto objects for the duration of the build, and some of those objects may be temporary that should not be persisted. In particular, UnixGlob and its inner classes should not outlive loading and analysis. For the most part, care was taken in this CL to only use strings that required no additional construction, mainly to minimize garbage (retaining references to newly created strings is not as great a concern since only the strings corresponding to the slowest K tasks are retained, for some relatively small values of K). Action descriptions for actually executing actions are eagerly expanded because that work is minimal compared to the work of actually executing an action. PiperOrigin-RevId: 191251488
* Replace path implementation.Gravatar tomlu2018-02-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Path and PathFragment have been replaced with String-based implementations. They are pretty similar, but each method is dissimilar enough that I did not feel sharing code was appropriate. A summary of changes: PATH ==== * Subsumes LocalPath (deleted, its tests repurposed) * Use a simple string to back Path * Path instances are no longer interned; Reference equality will no longer work * Always normalized (same as before) * Some operations will now be slower, like instance compares (which were previously just a reference check) * Multiple identical paths will now consume more memory since they are not interned PATH FRAGMENT ============= * Use a simple string to back PathFragment * No more segment arrays with interned strings * Always normalized * Remove isNormalized * Replace some isNormalizied uses with containsUpLevelReferences() to check if path fragments try to escape their scope * To check if user input is normalized, supply static methods on PathFragment to validate the string before constructing a PathFragment * Because PathFragments are always normalized, we have to replace checks for literal "." from PathFragment#getPathString to PathFragment#getSafePathString. The latter returns "." for the empty string. * The previous implementation supported efficient segment semantics (segment count, iterating over segments). This is now expensive since we do longer have a segment array. ARTIFACT ======== * Remove Path instance. It is instead dynamically constructed on request. This is necessary to avoid this CL becoming a memory regression. RELNOTES: None PiperOrigin-RevId: 185062932
* Automated rollback of commit 80c8ff1819a889814cbe2dae477d7fedce6aa181.Gravatar tomlu2018-01-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Rolling forward after underlying issue has been fixed. *** Original change description *** Automated rollback of commit d50cbbeef115f28c0cea1ac17572e0f12c0cf312. *** Reason for rollback *** b/71442447 *** Original change description *** Remove synchronization from file system. After the path refactor we will no longer have path instances to synchronize on. The underlying OS file systems are already naturally thread safe, that is, their internal data structures cannot be damaged. Any further synchronization (eg. races between directory creation and deletion) has to be managed at the client level. The l... *** PiperOrigin-RevId: 181368707
* Automated rollback of commit cb7689404ef9a69488d64bfc2e0bfb9326e664d6.Gravatar tomlu2018-01-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Rolling forward after the underlying issue has been fixed. *** Original change description *** Automated rollback of commit 6a54339bb943702bd7dffc43f85267dac98dc355. *** Reason for rollback *** b/71442447 *** Original change description *** Call through to Path#createDirectoryAndParents from FileUtils. This CL removes a method that due to its implementation causes threading difficulties for Path#createDirectory. The tests for the method are brought across to FileSystemTests since the methods are now implemented natively by the FileSystem classes. The tests were also cleaned up. The test revealed an edge c... *** PiperOrigin-RevId: 181367850
* Automated rollback of commit 6a54339bb943702bd7dffc43f85267dac98dc355.Gravatar tomlu2018-01-05
| | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** b/71442447 *** Original change description *** Call through to Path#createDirectoryAndParents from FileUtils. This CL removes a method that due to its implementation causes threading difficulties for Path#createDirectory. The tests for the method are brought across to FileSystemTests since the methods are now implemented natively by the FileSystem classes. The tests were also cleaned up. The test revealed an edge case bug in JavaIoFileSystem, so fix this. In two cases some code was using the return value from the old method. Returning "f... *** ROLLBACK_OF=179864042 PiperOrigin-RevId: 180946251
* Automated rollback of commit d50cbbeef115f28c0cea1ac17572e0f12c0cf312.Gravatar tomlu2018-01-05
| | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** b/71442447 *** Original change description *** Remove synchronization from file system. After the path refactor we will no longer have path instances to synchronize on. The underlying OS file systems are already naturally thread safe, that is, their internal data structures cannot be damaged. Any further synchronization (eg. races between directory creation and deletion) has to be managed at the client level. The last attempt to do this failed because of races in FileUtils#createDirectoryAndParents on Windows. This method is now gone, rep... *** ROLLBACK_OF=180290901 PiperOrigin-RevId: 180936132
* Remove synchronization from file system.Gravatar tomlu2017-12-28
| | | | | | | | | | After the path refactor we will no longer have path instances to synchronize on. The underlying OS file systems are already naturally thread safe, that is, their internal data structures cannot be damaged. Any further synchronization (eg. races between directory creation and deletion) has to be managed at the client level. The last attempt to do this failed because of races in FileUtils#createDirectoryAndParents on Windows. This method is now gone, replaced by a method from the Java framework that knows how to synchronize. PiperOrigin-RevId: 180290901
* Call through to Path#createDirectoryAndParents from FileUtils.Gravatar tomlu2017-12-21
| | | | | | | | | | | | This CL removes a method that due to its implementation causes threading difficulties for Path#createDirectory. The tests for the method are brought across to FileSystemTests since the methods are now implemented natively by the FileSystem classes. The tests were also cleaned up. The test revealed an edge case bug in JavaIoFileSystem, so fix this. In two cases some code was using the return value from the old method. Returning "false" essentially means that the directory already existed, and the code doesn't look racy, so we can replace it with an existence check. PiperOrigin-RevId: 179864042
* Add FileSystem#createDirectoryAndParents.Gravatar tomlu2017-12-21
| | | | | | | | A native implementation of this (instead of using FileSystemUtils, which can only use public interfaces) should be more efficient and more easy to make correct. In particular, it should allow removing FileSystemUtils#createDirectoriesAndParents, which has poor thread safety characteristics. The latter method has a lot of logic that forces certain unnatural atomicity guarantees on createDirectory, and it also has logic that is conditional on sub-string content of exception messages. PiperOrigin-RevId: 179819623
* Automated rollback of commit 82e68b75304438c96ff878a0c2b8d18b42002486.Gravatar aehlig2017-12-19
| | | | | | | | | | | | | | Fixes #4322, #4306. *** Reason for rollback *** Introduces a deadlock (see https://github.com/bazelbuild/bazel/issues/4322) *** Original change description *** Make FileSystem operate on LocalPath instead of Path. PiperOrigin-RevId: 179549866
* Make FileSystem operate on LocalPath instead of Path.Gravatar tomlu2017-12-14
| | | | PiperOrigin-RevId: 179082062
* Thread FileSystem instance through a few call sites.Gravatar tomlu2017-12-11
| | | | PiperOrigin-RevId: 178704585
* Thread filesystem through codebase.Gravatar tomlu2017-12-08
| | | | | RELNOTES: None PiperOrigin-RevId: 178426166
* Refactor the FileSystem API to allow for different hash functions.Gravatar buchgr2017-11-30
| | | | | | | | | | | | | Refactor the FileSystem class to include the hash function as an instance field. This allows us to have a different hash function per FileSystem and removes technical debt, as currently that's somewhat accomplished by a horrible hack that has a static method to set the hash function for all FileSystem instances. The FileSystem's default hash function remains MD5. RELNOTES: None PiperOrigin-RevId: 177479772
* [Trivial] Add a designated method for retrieving java.nio.file.PathGravatar Taras Tsugrii2017-10-30
| | | | | | | | | | | As explained in the javadoc this avoids unnecessary allocations and preserves underlying Java filesystem information. This partially addresses #3952. Closes #3953. PiperOrigin-RevId: 173877045
* Automated rollback of commit 5cca89f970bcaddb98972055f2d9539e9d225e67.Gravatar pcloudy2017-10-24
| | | | | | | | | | | | | *** Reason for rollback *** Break CI Windows nightly build: https://github.com/bazelbuild/bazel/issues/3927 https://ci.bazel.io/blue/organizations/jenkins/Global%2Fbazel-tests/detail/bazel-tests/314/pipeline/31 *** Original change description *** Remove synchronization from FileSystem implementations. PiperOrigin-RevId: 173243523
* Change FileSystem#getDirectoryEntries to return strings of the ↵Gravatar tomlu2017-10-23
| | | | | | | | file/directory names instead of paths. This is a small isolated change that can be done ahead of the big refactoring. PiperOrigin-RevId: 173124518
* Add Path argument to supportsModifications, supports*LinksNatively.Gravatar tomlu2017-10-20
| | | | | | | | This works better with UnionFileSystem, as (eg.) different mapped file systems might not agree on whether they are read-only. No actual code changes have been made that actually uses this argument, so this should be a behaviour no-op. PiperOrigin-RevId: 172782759
* Remove synchronization from FileSystem implementations.Gravatar tomlu2017-10-12
| | | | PiperOrigin-RevId: 171905267
* More BUILD file refactorings.Gravatar philwo2017-09-06
| | | | | | | | | Split collect, concurrent, vfs, windows into package-level BUILD files. Move clock classes out of "util", into their own Java package. Move CompactHashSet into its own Java package to break a dependency cycle. Give nestedset and inmemoryfs their own package-level BUILD files. PiperOrigin-RevId: 167702127
* Adding support for SHA256 for remote execution. Switch remote execution to useGravatar olaola2017-06-20
| | | | | | | | | the currently defined hash function for blobs. Some refactoring. Adding an option to set the hash function in the remote worker, defaulting to the current behavior (unfortunately it is a build option, have not found a clean way to specify it at runtime). BUG=62622420 TESTED=remote worker RELNOTES: none PiperOrigin-RevId: 159473116
* Refactor all ctor callsites of PathFragment to instead call a static ↵Gravatar nharmata2017-04-05
| | | | | | | | | | | | 'create' method. This paves the way for changing PathFragment to e.g. an abstract class with multiple subclasses. This way we can split out the windows-specific stuff into one of these concrete classes, making the code more readable and also saving memory (since the shallow heap size of the NonWindowsPathFragment subclass will hopefully be smaller than that of the current PathFragment). This also lets us pursue gc churn optimizations. We can now do interning in PathFragment#create and can also get rid of unnecessary intermediate PathFragment allocations. RELNOTES: None PiperOrigin-RevId: 152145768
* Move UnixFileSystem to lib.unix, WindowsFileSystem to lib.windowsGravatar Ulf Adams2017-02-28
| | | | | | -- PiperOrigin-RevId: 148749485 MOS_MIGRATED_REVID=148749485
* Move FileAccessException to the vfs packageGravatar Ulf Adams2017-02-17
| | | | | | | | | | | | | | It was previously in unix, but also used from non-unix file systems, which means it's not actually unix-specific. This is in preparation for splitting compilation of the unix and windows file systems into separate libraries. That improves layering and reduces compile times - note that Bazel already injects the vfs into its lower layers, which should only rely on the normal vfs APIs, not on anything platform-specific. -- PiperOrigin-RevId: 147829659 MOS_MIGRATED_REVID=147829659
* Fixed the issue that hard links are handled improperly when bazel ↵Gravatar Googler2016-09-08
| | | | | | | | | decompresses tarballs. Issue link: https://github.com/bazelbuild/bazel/issues/574 -- MOS_MIGRATED_REVID=132434278
* Fixed JavaIoFileSystem.setLastModifiedTime to actually match its documented ↵Gravatar Chris Parsons2016-07-27
| | | | | | | behavior (passing -1 modified time should use system time). -- MOS_MIGRATED_REVID=128489592
* Support case-insensitive comparision in Path.java with WindowsFileSystemGravatar Yun Peng2016-05-09
| | | | | | | | Since file path is case-insensitive on Windows, we need to support this. Also fixed .d file inclusions check in CppCompileAction.java on Windows -- MOS_MIGRATED_REVID=121823250
* Merge Dmitry's jury-rigged WindowsFileSystem to decrease the number of ↵Gravatar Lukacs Berki2016-01-29
| | | | | | | | | changes between his branch and HEAD. This code will need some JNI to make it work properly, but for now, it will do. -- MOS_MIGRATED_REVID=113349143
* Make local_repository and new_local_repository work with the limitations of ↵Gravatar Lukacs Berki2016-01-26
| | | | | | | our plans for symlink support on Windows. -- MOS_MIGRATED_REVID=113043269
* Make AbstractFileSystem correct in isolation. Previously there was ↵Gravatar Nathan Harmata2015-11-19
| | | | | | | | | (accidentally) the implicit requirement that the deriving class override FileSystem#stat. Even though this wasn't a problem in practice in the Bazel codebase (since all of our transitive subclasses had custom 'stat' implementation), it's good hygiene to have things correct (e.g. if we added a new subclass). -- MOS_MIGRATED_REVID=108158039
* Introduce Path#isSpecialFile, FileSystem#isSpecialFile, and ↵Gravatar Nathan Harmata2015-10-21
| | | | | | | FileStatus#isSpecialFile to help disambiguate between a regular file and a special file, since the file size of a special file cannot be trusted. -- MOS_MIGRATED_REVID=105903622
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Use VFS_READLINK rather than VFS_LINK, to avoid confusion with theGravatar Han-Wen Nienhuys2015-03-13
| | | | | | | link system call. -- MOS_MIGRATED_REVID=88441395
* Update from Google.Gravatar Han-Wen Nienhuys2015-02-25
-- MOE_MIGRATED_REVID=85702957
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-13 09:28:40 +0000