aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/vfs
Commit message (Collapse)AuthorAge
* Thread FileSystem through to a single Path#delete call site.Gravatar tomlu2017-10-30
| | | | | | This requires a fairly large amount of changes to fundamental objects like BlazeRuntime, Executor, and so on, as well as changing a lot of test code to thread the file system through. I expect future CLs to be much smaller. PiperOrigin-RevId: 173678144
* Automated rollback of commit 5cca89f970bcaddb98972055f2d9539e9d225e67.Gravatar pcloudy2017-10-24
| | | | | | | | | | | | | *** Reason for rollback *** Break CI Windows nightly build: https://github.com/bazelbuild/bazel/issues/3927 https://ci.bazel.io/blue/organizations/jenkins/Global%2Fbazel-tests/detail/bazel-tests/314/pipeline/31 *** Original change description *** Remove synchronization from FileSystem implementations. PiperOrigin-RevId: 173243523
* Change FileSystem#getDirectoryEntries to return strings of the ↵Gravatar tomlu2017-10-23
| | | | | | | | file/directory names instead of paths. This is a small isolated change that can be done ahead of the big refactoring. PiperOrigin-RevId: 173124518
* Fix UnionFileSystem's implementation of supportsModifications and friends.Gravatar tomlu2017-10-23
| | | | | | It now correctly delegates to the right file system. PiperOrigin-RevId: 172916854
* Lazy-initialise rootPath in FileSystem.Gravatar tomlu2017-10-23
| | | | | | Path construction entails calling isFileSystemCaseSensitive at a time when subclasses have not been initialised. This can cause a crash. Solve by deferring init of the path. PiperOrigin-RevId: 172914501
* Remove scope escaping capability from InMemoryFileSystem.Gravatar tomlu2017-10-23
| | | | | | | | Nobody sets a delegate anymore, meaning it should be fine to crash hard if we try to resolve a file outside scope. The CL was created by inlining ScopeEscapableFileSystem into InMemoryFileSystem, making it impossible to return an outOfScope file status (instead throwing an exception), then trivially fixing the code. PiperOrigin-RevId: 172886769
* Make UnionFileSystem eagerly resolve symlinks.Gravatar tomlu2017-10-20
| | | | | | This allows symlinks between multiple different file systems. InMemoryFileSystem uses this in some tests. This CL will allow deletion of ScopeEscapableFileSystem, one of two file systems that partially duplicate what UnionFileSystem does. PiperOrigin-RevId: 172823391
* Add Path argument to supportsModifications, supports*LinksNatively.Gravatar tomlu2017-10-20
| | | | | | | | This works better with UnionFileSystem, as (eg.) different mapped file systems might not agree on whether they are read-only. No actual code changes have been made that actually uses this argument, so this should be a behaviour no-op. PiperOrigin-RevId: 172782759
* Remove synchronization from FileSystem implementations.Gravatar tomlu2017-10-12
| | | | PiperOrigin-RevId: 171905267
* Add follow-symlinks parameter to FileSystemUtils#copyTreesBelow, so that all ↵Gravatar janakr2017-10-12
| | | | | | the tool setup we do in tests doesn't necessarily have to be copied if we copy a workspace over. PiperOrigin-RevId: 171864170
* Clean up TODOs.Gravatar László Csomor2017-10-09
| | | | | | | | | | | | | | | | | | | In this commit: * buildenv.sh: restore its state to that as of commit 511c35b46cead500d4e76706e0a709e50995ceba * CommonCommandOptions.java: remove a deprecated no-op flag * WindowsPathFragment.java: implement an ASCII-only isLetter function, use that instead of Character.isLetter, because the latter returns true for some Unicode characters too * bazel_bootstrap_distfile_test: remove logging that we no longer need, since the bugfix for issue #3618 will be pushed to GitHub later today Change-Id: Ibda70219e974f0c47bc82addc647d8951f4bd701 PiperOrigin-RevId: 171498977
* Make Fdo use straight zip files, delete ZipFileSystem.Gravatar tomlu2017-10-06
| | | | PiperOrigin-RevId: 171221156
* Make UnionFileSystem accept all paths Bazel can throw at it.Gravatar ccalvarin2017-09-27
| | | | | | | | | Instead of relying on a character-by-character StringTrie, segment paths based on PathFragments. This means UnionFS can accept any path that Bazel stores internally, removing the ASCII limitations. This also means removing the ability to have a filesystem bound for sub-PathFragments, /foo/barbar, /foo/barqux could have the same filesystem bound at /foo/bar. This feature was tested for when a use case was envisioned, but it was never used, so removing it is safe. RELNOTES: None. PiperOrigin-RevId: 170054656
* bep: Properly URI encode file pathsGravatar buchgr2017-09-14
| | | | | | RELNOTES: The Build Event Protocol's File.uri field is now properly encoded according to RFC2396. PiperOrigin-RevId: 168674865
* More BUILD file refactorings.Gravatar philwo2017-09-06
| | | | | | | | | Split collect, concurrent, vfs, windows into package-level BUILD files. Move clock classes out of "util", into their own Java package. Move CompactHashSet into its own Java package to break a dependency cycle. Give nestedset and inmemoryfs their own package-level BUILD files. PiperOrigin-RevId: 167702127
* Fix assorted ErrorProne warnings.Gravatar lberki2017-09-06
| | | | | RELNOTES: None. PiperOrigin-RevId: 167574104
* Rename all logger instances to "logger" (instead "LOG" or "log").Gravatar lberki2017-09-05
| | | | | RELNOTES: None. PiperOrigin-RevId: 167505493
* Close the ZipFileSystem and the underlying ZipFile appropriately after we ↵Gravatar lberki2017-07-26
| | | | | | | | | | | finished extracting the FDO profile. Also fix a truly embarrassing infinite recursion bug introduced by Yours Truly in unknown commit . This avoids a failure mode where, when two profiles at the same path are used in two builds close one after the other, the file handle would get erroneously re-used. RELNOTES: None. PiperOrigin-RevId: 163063976
* Add hook to process SkyframeExecutor in tests. Also add sentinel exception ↵Gravatar janakr2017-07-25
| | | | | | to indicate unserializability, improve error message in Path when serialization fails, and add some test-only methods to SkyframeExecutor and PackageFactory. PiperOrigin-RevId: 162993806
* Add all generated headers as mandatory inputs for compile actions.Gravatar rduan2017-07-21
| | | | | | | We previously only add those under genfiles root. But J2ObjC and objc proto headers are under the bin root, which we may also fix in the near future. RELNOTES: None. PiperOrigin-RevId: 162656166
* Inline AttributeContainer.ATTRIBUTE_CONTAINER_FACTORY and ↵Gravatar laurentlb2017-07-03
| | | | | | | PathFragment.TO_PATH_FRAGMENT RELNOTES: None. PiperOrigin-RevId: 160668541
* Automated conversion to Java 8Gravatar laurentlb2017-06-30
| | | | | | | With a few manual fixes for readability. RELNOTES: None. PiperOrigin-RevId: 160582556
* Allow "[", "]", "{", and "}" in globs in BUILD files. (#3048)Gravatar Jonathan Bluett-Duncan2017-06-22
| | | | | | | | | | | | PR #2679 allowed parentheses ("(" and ")") in globs, but other bracket style characters were not allowed at the time. It was decided in issues #2583 and #3048 that other bracket characters should be allowed as well, since there is no apparent historical reason for disallowing them in the first place. Closes #3166. PiperOrigin-RevId: 159691498
* Adding support for SHA256 for remote execution. Switch remote execution to useGravatar olaola2017-06-20
| | | | | | | | | the currently defined hash function for blobs. Some refactoring. Adding an option to set the hash function in the remote worker, defaulting to the current behavior (unfortunately it is a build option, have not found a clean way to specify it at runtime). BUG=62622420 TESTED=remote worker RELNOTES: none PiperOrigin-RevId: 159473116
* Cleanup: addressing comments from commit ↵Gravatar olaola2017-06-16
| | | | | | | 9eea05d068a06ab642dd9d86d46ee5fa2e36b02e. RELNOTES: n/a PiperOrigin-RevId: 158988688
* For objc rules, generated headers are mandatory inputs to CppCompileAction.Gravatar cpeyser2017-05-26
| | | | PiperOrigin-RevId: 157218175
* Remove stale comment and add back a removed TODO.Gravatar nharmata2017-04-28
| | | | | RELNOTES: None PiperOrigin-RevId: 154543320
* Make PathFragment an abstract class.Gravatar nharmata2017-04-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are now four concrete implementations: RelativeUnixPathFragment, AbsoluteUnixPathFragment, RelativeWindowsPathFragment, AbsoluteWindowsPathFragment. Goals: -Reduce memory usage of PathFragment on non-Windows platforms while maintaining existing semantics. -Make a few simple performance improvements along the way. -Add TODOs for a few more simple performance improvements. -Open the way for reducing code complexity of PathFragment. All of the Windows-specific stuff ought not complicate the code so much. Non goals: -Make the entire codebase as pretty as possible wrt PathFragment & Windows. -Make PathFragment usage more sane in general (e.g. change semantics to ban coexistence of Windows and Unix PathFragments). -Optimize PathFragment as much as possible wrt memory or even in any other dimensions (e.g. gc churn, cpu). To elaborate, the primary motivation is per-instance memory usage of PathFragment on Unix platforms: Before this change ------------------ +UseCompressedOops --> 32 bytes per instance -UseCompressedOops --> 40 bytes per instance After this change ------------------ +UseCompressedOops --> 24 bytes per instance -UseCompressedOops --> 32 bytes per instance Since Bazel can retain lots of PathFragments, the memory savings of this CL are fairly large. RELNOTES: None PiperOrigin-RevId: 154052905
* Migrate UnixGlob to Path#statIfFound() instead of #statNullable(). The ↵Gravatar felly2017-04-18
| | | | | | | | | latter swallows all filesystem failures, and does not disambiguate missing files from filesystem problems. The syscall cache now tracks IOExceptions if they are present, just as it does with readdir(). RELNOTES: None PiperOrigin-RevId: 153185433
* fix: Allow parentheses in glob patternsGravatar Harmandeep Kaur2017-04-11
| | | | | | | More info here #2583 Closes #2679. PiperOrigin-RevId: 152685327
* Update stale Windows path comment.Gravatar ccalvarin2017-04-11
| | | | PiperOrigin-RevId: 152684266
* Remove the last remaining public ctor from PathFragment.Gravatar nharmata2017-04-07
| | | | | RELNOTES: None PiperOrigin-RevId: 152400979
* Refactor all ctor callsites of PathFragment to instead call a static ↵Gravatar nharmata2017-04-05
| | | | | | | | | | | | 'create' method. This paves the way for changing PathFragment to e.g. an abstract class with multiple subclasses. This way we can split out the windows-specific stuff into one of these concrete classes, making the code more readable and also saving memory (since the shallow heap size of the NonWindowsPathFragment subclass will hopefully be smaller than that of the current PathFragment). This also lets us pursue gc churn optimizations. We can now do interning in PathFragment#create and can also get rid of unnecessary intermediate PathFragment allocations. RELNOTES: None PiperOrigin-RevId: 152145768
* Restructure Path#getCachedChildPath to make direct method calls without ↵Gravatar nharmata2017-04-04
| | | | | | | | allocating a garbage TranslatedPath object. RELNOTES: None PiperOrigin-RevId: 152130170
* Make InMemoryFileSystem case-insensitive on Windows. Gravatar Dmitry Lomov2017-03-09
| | | | | | | | | | This fixes PackagePathLocatorTest. -- Change-Id: I3d14a80993f6b256acfc732adf2d97b1d2dcb804 Reviewed-on: https://cr.bazel.build/9310 PiperOrigin-RevId: 149634730 MOS_MIGRATED_REVID=149634730
* PathFragment comparisons are now platform-aware Gravatar László Csomor2017-03-06
| | | | | | | | | | | | | | | | | | | | | | | | | PathFragment's `equals`, `hashCode`, `compareTo`, `startsWith`, `endsWith`, and `relativeTo` are now aware of case-insensitivity when running on Windows. This approach is better than https://bazel-review.googlesource.com/c/9124/ because it preserves path casing, which is important when computing action output paths. This change contains two additional bugfixes: - `compareTo` now takes `driveLetter` into account - the `InMemoryFileSystem` in `PathWindowsTest` is not case-insensitive Fixes https://github.com/bazelbuild/bazel/issues/2613 -- Change-Id: I1a4250a373fff03fa02a6d8360457450b47a42a8 Reviewed-on: https://cr.bazel.build/9126 PiperOrigin-RevId: 149106930 MOS_MIGRATED_REVID=149106930
* Move UnixFileSystem to lib.unix, WindowsFileSystem to lib.windowsGravatar Ulf Adams2017-02-28
| | | | | | -- PiperOrigin-RevId: 148749485 MOS_MIGRATED_REVID=148749485
* Remove the deprecated com.google.devtools.build.lib.vfs.FileSystems classGravatar Ulf Adams2017-02-27
| | | | | | -- PiperOrigin-RevId: 148625618 MOS_MIGRATED_REVID=148625618
* Move FileAccessException to the vfs packageGravatar Ulf Adams2017-02-17
| | | | | | | | | | | | | | It was previously in unix, but also used from non-unix file systems, which means it's not actually unix-specific. This is in preparation for splitting compilation of the unix and windows file systems into separate libraries. That improves layering and reduces compile times - note that Bazel already injects the vfs into its lower layers, which should only rely on the normal vfs APIs, not on anything platform-specific. -- PiperOrigin-RevId: 147829659 MOS_MIGRATED_REVID=147829659
* Windows: use JNI CreateJunction in Java code Gravatar Laszlo Csomor2017-02-15
| | | | | | | | | | | | | | | | Use the new CreateJunction in the Windows JNI code every time we need to create junctions. This means updating WindowsFileOperations and related tests. Add test for WindowsFileSystem.createSymbolicLink. See https://github.com/bazelbuild/bazel/issues/2238 -- Change-Id: I5827e2e70e8e147f5f102fabf95fa9a148b3bcdc Reviewed-on: https://cr.bazel.build/8896 PiperOrigin-RevId: 147598107 MOS_MIGRATED_REVID=147598107
* WindowsFileSystem.java: Fix createSymbolicLink when target is a relative path Gravatar Yun Peng2017-02-07
| | | | | | | | | | Fixed https://github.com/bazelbuild/bazel/issues/2494 -- Change-Id: I2ca335fa5b3a7759f57085717128912f09363789 Reviewed-on: https://cr.bazel.build/8650 PiperOrigin-RevId: 146761747 MOS_MIGRATED_REVID=146761747
* The double-checked locking seems unnecessary.Gravatar Googler2017-02-06
| | | | | | | | | | | | | | | Every double-checked lock will be followed up by a regular lock. Either the children don't exist, in which case we take the lock twice, or they do exist and we take the lock once. This makes it so we only ever take the lock once. I don't know if this makes it faster, but it makes the code simpler, and shouldn't make it slower. -- PiperOrigin-RevId: 146553235 MOS_MIGRATED_REVID=146553235
* Windows: resolve shortened pathsGravatar Laszlo Csomor2017-01-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | TL;DR: resolve "C:/progra~1" style paths, don't cache failed resolutions in the parent's Path.children When creating WindowsPath objects, resolve the "C:/progra~1" style paths to "C:/program files". This enables us to correctly compare paths on Windows. Without this canonicalization we incorrectly determine "C:/progra~1" and "C:/program files" to be different when they are actually the same. We only attempt to resolve such paths if the name looks like it's an abbreviated path. If resolution fails, probably due to the path not existing, then we don't cache this Path object in the parent's `children` list. This avoids stale cache entries in case the path springs into existence later in the server's lifetime. Fixes https://github.com/bazelbuild/bazel/issues/2145 We also need to rectify https://github.com/bazelbuild/bazel/issues/2173 -- PiperOrigin-RevId: 143442134 MOS_MIGRATED_REVID=143442134
* WindowsFileSystem: use JNI impl. of isJunctionGravatar Laszlo Csomor2016-12-20
| | | | | | | | | | | | | | | | | | | Also build the JNI library while bootstraping. This was once submitted in commit 4a249b6962d32ed4cfd4165dfdae4a555b00bb69 but got rolled back due to some test breakage that's long since fixed. In this change I'm slightly modifying the original code in compile.sh. Using JNI methods however is necessary because we can't implement WindowsFileOperations.GetLongPath in native Java, and having that code is a prerequisite for the fix of https://github.com/bazelbuild/bazel/issues/2145 See also https://github.com/bazelbuild/bazel/issues/2238 -- PiperOrigin-RevId: 142535019 MOS_MIGRATED_REVID=142535019
* Bazel client, Windows: implement AsWindowsPathGravatar Laszlo Csomor2016-12-20
| | | | | | | | | | | | | | | | | | | | This method converts MSYS paths to Windows path. It uses the BAZEL_SH envvar to obtain the MSYS root directory, to which all Unix paths (except for mounts) are relative. We cannot handle mounts because we don't want to read /etc/mtab every time there's a file operation so we simply apply a heuristic similar to https://github.com/bazelbuild/bazel/blob/cd4cc09fa6ef96380a3d0888f825dfd1dbada651/src/main/java/com/google/devtools/build/lib/vfs/WindowsFileSystem.java#L52-L63 Also clean up the #ifdefs surrounding SyncFile. See https://github.com/bazelbuild/bazel/issues/2107 -- PiperOrigin-RevId: 142531986 MOS_MIGRATED_REVID=142531986
* Fix PathFragment to not use Java8-only static hashCode methods.Gravatar John Cater2016-12-15
| | | | | | | | Fixes #2247. -- PiperOrigin-RevId: 142143520 MOS_MIGRATED_REVID=142143520
* Replace UnixGlob's package glob prefetching functionality with anGravatar Googler2016-12-02
| | | | | | | | optional method in FileSystem. Custom FileSystem implementations can use this to provide their own implementation of glob prefetching. -- MOS_MIGRATED_REVID=140736304
* Introduce BlazeInterners, a Blaze-specific wrapper around Guava's Interners ↵Gravatar Nathan Harmata2016-11-24
| | | | | | | | | | | that makes an appropriate call to Interners.InternerBuilder#concurrencyLevel. For current readers of this CL, I used this class everywhere in the Blaze codebase. For future readers of this CL, this class should be used to create an Interner in the Blaze codebase. -- MOS_MIGRATED_REVID=140063271
* Adding an option to set the digest function that everything uses. Minor ↵Gravatar Ola Rozenfeld2016-11-18
| | | | | | | | | | | | | refactoring: enabling potential fast digest computation of more than one digest function type. Usage: bazel --host_jvm_args="-Dbazel.DigestFunction=SHA1" build ... Ugliness: using a system property (a static non-final variable), because the better way to do it (a flag) would result in a much, much larger refactoring. More ugliness: I have updated the minimal amount of tests. A lot of tests are still relying on the default value of MD5. Ideally, they need to be updated as well. -- MOS_MIGRATED_REVID=139490836
* Windows: can now remove bazel-* symlinksGravatar Laszlo Csomor2016-11-04
| | | | | | | | | | | | | | | | | This change updates WindowsFileSystem so it: - retrieves the DosFileAttributes instead of the BasicFileAttributes, because the latter does not report junctions as directories - uses just isJunction to decide if a file is a symlink, doesn't look at whether it's a directory (again because java.nio.File also incorrectly reports junctions as non-directories) Fixes https://github.com/bazelbuild/bazel/issues/1850 -- MOS_MIGRATED_REVID=138187220
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-24 19:29:03 +0000