aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/vfs/Path.java
Commit message (Collapse)AuthorAge
* Add easy access to MessageDigests in DigestHashFunctionGravatar ccalvarin2018-08-13
| | | | | | | Move the message-digest cloning to DigestHashFunction and out of Fingerprint, to make it possible to configure Fingerprint to use different hash functions. We keep the default MD5 for now, we'd like it to use the global default but want to isolate the configuration change from any change adding potential contention. RELNOTES: None. PiperOrigin-RevId: 208502993
* Remove redundancy in DigestHashFunction use in FileSystem.Gravatar ccalvarin2018-07-23
| | | | | | | Each FileSystem instance has a digest function, but the getDigest and getHashDigest functions also accepted their own custom parameter functions. We only support 1 hash per filesystem instance, these parameters are redundant. RELNOTES: None. PiperOrigin-RevId: 205758571
* Add a utility function to hash a directoryGravatar Klaus Aehlig2018-07-20
| | | | | | | | | | Return a hash of a directory that is suitable to verify whether a repository rule contained a good snapshot of source code. So certain aspects of the directory, in particular ownership of the files, are deliberately not included in the hash. Change-Id: I1b35f7af47b376808acad3b6e54daaaec4f9ebfd PiperOrigin-RevId: 205382020
* Move HashFunction out of FileSystem, and turn it into a class, instead of an ↵Gravatar ccalvarin2018-06-21
| | | | | | | | | enum. Now that we aren't using enum names for the hash functions, we also accept the standard names, such as SHA-256. RELNOTES: None. PiperOrigin-RevId: 201624286
* Serialize Path using AutoCodec.Gravatar janakr2018-03-05
| | | | PiperOrigin-RevId: 187945746
* Deletes CODEC fields now that they are no longer needed.Gravatar shahan2018-02-28
| | | | PiperOrigin-RevId: 187397314
* Replaces InjectingObjectCodec with dependencies threaded through ↵Gravatar shahan2018-02-13
| | | | | | (Des|S)erializationContext. PiperOrigin-RevId: 185547740
* Add context argument to ObjectCodec.{serialize,deserialize}Gravatar michajlo2018-02-11
| | | | | | | | | | Context implementations are currently empty, just doing the plumbing in this change. Once this is in we can start passing along the ObjectCodecRegistry, which will allow runtime codec resolution for classes not known at compile time. We'll also inevitably add some memoization helpers, allowing us to optimize the serialization process further. PiperOrigin-RevId: 185305674
* Replace path implementation.Gravatar tomlu2018-02-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Path and PathFragment have been replaced with String-based implementations. They are pretty similar, but each method is dissimilar enough that I did not feel sharing code was appropriate. A summary of changes: PATH ==== * Subsumes LocalPath (deleted, its tests repurposed) * Use a simple string to back Path * Path instances are no longer interned; Reference equality will no longer work * Always normalized (same as before) * Some operations will now be slower, like instance compares (which were previously just a reference check) * Multiple identical paths will now consume more memory since they are not interned PATH FRAGMENT ============= * Use a simple string to back PathFragment * No more segment arrays with interned strings * Always normalized * Remove isNormalized * Replace some isNormalizied uses with containsUpLevelReferences() to check if path fragments try to escape their scope * To check if user input is normalized, supply static methods on PathFragment to validate the string before constructing a PathFragment * Because PathFragments are always normalized, we have to replace checks for literal "." from PathFragment#getPathString to PathFragment#getSafePathString. The latter returns "." for the empty string. * The previous implementation supported efficient segment semantics (segment count, iterating over segments). This is now expensive since we do longer have a segment array. ARTIFACT ======== * Remove Path instance. It is instead dynamically constructed on request. This is necessary to avoid this CL becoming a memory regression. RELNOTES: None PiperOrigin-RevId: 185062932
* Adds FileSystem check to Path.CODEC.serialize.Gravatar shahan2018-01-12
| | | | PiperOrigin-RevId: 181797078
* Adds a CODEC for CppConfiguration.Gravatar shahan2018-01-10
| | | | | | | * Creates an enum for cpu transformer, which is easier to serialize than an opaque function. This also means moving FakeCPU to avoid introducing a circular dependency. * Adds a CODEC to Path using InjectingObjectCodec. PiperOrigin-RevId: 181445911
* Undo change to introduce methods to remove file system instance from Path.Gravatar tomlu2017-12-28
| | | | | | | | I no longer believe that it is worthwhile to remove the file system instance from Path. Measurements indicate that I can simply replace the Path implementation in one go and just leave the instance on the Path class. This CL was done entirely using IDE refactoring, so should be safe. PiperOrigin-RevId: 180287162
* Redo FileType to reduce generated garbage.Gravatar tomlu2017-12-22
| | | | | | | | | | | | | * Change FileType to no longer assume it operates on just the base name (it can now be given a full path). * Move the responsibility to specific classes (Artifact, Path, PathFragment) to decide how they want to offer up a string that includes the file name. * Flip the order in which users are expected to check Artifact type, from FileType#matches(Artifact) to Artifact#isFileType(FileType). This looks natural and should encourage developers to use efficient file type checking methods. * Change CppCompileAction to use the new API. RELNOTES: None PiperOrigin-RevId: 179903239
* Add FileSystem#createDirectoryAndParents.Gravatar tomlu2017-12-21
| | | | | | | | A native implementation of this (instead of using FileSystemUtils, which can only use public interfaces) should be more efficient and more easy to make correct. In particular, it should allow removing FileSystemUtils#createDirectoriesAndParents, which has poor thread safety characteristics. The latter method has a lot of logic that forces certain unnatural atomicity guarantees on createDirectory, and it also has logic that is conditional on sub-string content of exception messages. PiperOrigin-RevId: 179819623
* Automated rollback of commit 82e68b75304438c96ff878a0c2b8d18b42002486.Gravatar aehlig2017-12-19
| | | | | | | | | | | | | | Fixes #4322, #4306. *** Reason for rollback *** Introduces a deadlock (see https://github.com/bazelbuild/bazel/issues/4322) *** Original change description *** Make FileSystem operate on LocalPath instead of Path. PiperOrigin-RevId: 179549866
* Make FileSystem operate on LocalPath instead of Path.Gravatar tomlu2017-12-14
| | | | PiperOrigin-RevId: 179082062
* Move msys path support into DependencySet, removing it from the general Path ↵Gravatar tomlu2017-12-14
| | | | | | | | system. This is the only place that should actually need it. PiperOrigin-RevId: 179054861
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Break dependency on vfs from the interface of syntax and cmdline.Gravatar tomlu2017-11-06
| | | | | | | | These libs are exposed externally, implying that the vfs is also exposed externally. We break out PathFragment from vfs to still use this in their interface. This class is a much smaller dependency than the entire vfs. PiperOrigin-RevId: 174729373
* Add migration methods to Path.Gravatar tomlu2017-10-30
| | | | | | We add methods to Path that accept a file system instead of using the associated instance. These will be used during migration away from the methods that use the Path's file system. PiperOrigin-RevId: 173707291
* Thread FileSystem through to a single Path#delete call site.Gravatar tomlu2017-10-30
| | | | | | This requires a fairly large amount of changes to fundamental objects like BlazeRuntime, Executor, and so on, as well as changing a lot of test code to thread the file system through. I expect future CLs to be much smaller. PiperOrigin-RevId: 173678144
* Change FileSystem#getDirectoryEntries to return strings of the ↵Gravatar tomlu2017-10-23
| | | | | | | | file/directory names instead of paths. This is a small isolated change that can be done ahead of the big refactoring. PiperOrigin-RevId: 173124518
* Add Path argument to supportsModifications, supports*LinksNatively.Gravatar tomlu2017-10-20
| | | | | | | | This works better with UnionFileSystem, as (eg.) different mapped file systems might not agree on whether they are read-only. No actual code changes have been made that actually uses this argument, so this should be a behaviour no-op. PiperOrigin-RevId: 172782759
* bep: Properly URI encode file pathsGravatar buchgr2017-09-14
| | | | | | RELNOTES: The Build Event Protocol's File.uri field is now properly encoded according to RFC2396. PiperOrigin-RevId: 168674865
* Add hook to process SkyframeExecutor in tests. Also add sentinel exception ↵Gravatar janakr2017-07-25
| | | | | | to indicate unserializability, improve error message in Path when serialization fails, and add some test-only methods to SkyframeExecutor and PackageFactory. PiperOrigin-RevId: 162993806
* Automated conversion to Java 8Gravatar laurentlb2017-06-30
| | | | | | | With a few manual fixes for readability. RELNOTES: None. PiperOrigin-RevId: 160582556
* Adding support for SHA256 for remote execution. Switch remote execution to useGravatar olaola2017-06-20
| | | | | | | | | the currently defined hash function for blobs. Some refactoring. Adding an option to set the hash function in the remote worker, defaulting to the current behavior (unfortunately it is a build option, have not found a clean way to specify it at runtime). BUG=62622420 TESTED=remote worker RELNOTES: none PiperOrigin-RevId: 159473116
* Make PathFragment an abstract class.Gravatar nharmata2017-04-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are now four concrete implementations: RelativeUnixPathFragment, AbsoluteUnixPathFragment, RelativeWindowsPathFragment, AbsoluteWindowsPathFragment. Goals: -Reduce memory usage of PathFragment on non-Windows platforms while maintaining existing semantics. -Make a few simple performance improvements along the way. -Add TODOs for a few more simple performance improvements. -Open the way for reducing code complexity of PathFragment. All of the Windows-specific stuff ought not complicate the code so much. Non goals: -Make the entire codebase as pretty as possible wrt PathFragment & Windows. -Make PathFragment usage more sane in general (e.g. change semantics to ban coexistence of Windows and Unix PathFragments). -Optimize PathFragment as much as possible wrt memory or even in any other dimensions (e.g. gc churn, cpu). To elaborate, the primary motivation is per-instance memory usage of PathFragment on Unix platforms: Before this change ------------------ +UseCompressedOops --> 32 bytes per instance -UseCompressedOops --> 40 bytes per instance After this change ------------------ +UseCompressedOops --> 24 bytes per instance -UseCompressedOops --> 32 bytes per instance Since Bazel can retain lots of PathFragments, the memory savings of this CL are fairly large. RELNOTES: None PiperOrigin-RevId: 154052905
* Refactor all ctor callsites of PathFragment to instead call a static ↵Gravatar nharmata2017-04-05
| | | | | | | | | | | | 'create' method. This paves the way for changing PathFragment to e.g. an abstract class with multiple subclasses. This way we can split out the windows-specific stuff into one of these concrete classes, making the code more readable and also saving memory (since the shallow heap size of the NonWindowsPathFragment subclass will hopefully be smaller than that of the current PathFragment). This also lets us pursue gc churn optimizations. We can now do interning in PathFragment#create and can also get rid of unnecessary intermediate PathFragment allocations. RELNOTES: None PiperOrigin-RevId: 152145768
* Restructure Path#getCachedChildPath to make direct method calls without ↵Gravatar nharmata2017-04-04
| | | | | | | | allocating a garbage TranslatedPath object. RELNOTES: None PiperOrigin-RevId: 152130170
* The double-checked locking seems unnecessary.Gravatar Googler2017-02-06
| | | | | | | | | | | | | | | Every double-checked lock will be followed up by a regular lock. Either the children don't exist, in which case we take the lock twice, or they do exist and we take the lock once. This makes it so we only ever take the lock once. I don't know if this makes it faster, but it makes the code simpler, and shouldn't make it slower. -- PiperOrigin-RevId: 146553235 MOS_MIGRATED_REVID=146553235
* Windows: resolve shortened pathsGravatar Laszlo Csomor2017-01-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | TL;DR: resolve "C:/progra~1" style paths, don't cache failed resolutions in the parent's Path.children When creating WindowsPath objects, resolve the "C:/progra~1" style paths to "C:/program files". This enables us to correctly compare paths on Windows. Without this canonicalization we incorrectly determine "C:/progra~1" and "C:/program files" to be different when they are actually the same. We only attempt to resolve such paths if the name looks like it's an abbreviated path. If resolution fails, probably due to the path not existing, then we don't cache this Path object in the parent's `children` list. This avoids stale cache entries in case the path springs into existence later in the server's lifetime. Fixes https://github.com/bazelbuild/bazel/issues/2145 We also need to rectify https://github.com/bazelbuild/bazel/issues/2173 -- PiperOrigin-RevId: 143442134 MOS_MIGRATED_REVID=143442134
* Replace UnixGlob's package glob prefetching functionality with anGravatar Googler2016-12-02
| | | | | | | | optional method in FileSystem. Custom FileSystem implementations can use this to provide their own implementation of glob prefetching. -- MOS_MIGRATED_REVID=140736304
* Adding an option to set the digest function that everything uses. Minor ↵Gravatar Ola Rozenfeld2016-11-18
| | | | | | | | | | | | | refactoring: enabling potential fast digest computation of more than one digest function type. Usage: bazel --host_jvm_args="-Dbazel.DigestFunction=SHA1" build ... Ugliness: using a system property (a static non-final variable), because the better way to do it (a flag) would result in a much, much larger refactoring. More ugliness: I have updated the minimal amount of tests. A lot of tests are still relying on the default value of MD5. Ideally, they need to be updated as well. -- MOS_MIGRATED_REVID=139490836
* VFS: implement a Windows-specific Path subclassGravatar Laszlo Csomor2016-10-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change rolls forward commit e0d7a540e3c615c628f63fcaaaba0c47fca2cb25 and commit 8bb4299b28de14eed9d3b57bcaeb9350c81c7db3, and adds a bugfix: - FileSystem.PathFactory got a new translatePath method that WindowsFileSystem.PathFactory overrides to translate absolute Unix paths to MSYS-relative paths - Path.getCachedChildPath calls this translatePath method so the child path is registered with the correct (translated) parent and under the correct name (e.g. "C:" instead of say "c") Below is the rest of the original change description: The new subclass WindowsFileSystem.WindowsPath is aware of Windows drives. This change: - introduces a new factory for Path objects so FileSystems can return a custom implementation that instantiates filesystem-specific Paths - implements the WindowsPath subclass of Path that is aware of Windows drives - introduces the bazel.windows_unix_root JVM argument that defines the MSYS root, which defines the absolute Windows path that is the root of all Unix paths that Bazel creates (e.g. "/usr/lib" -> "C:/tools/msys64/usr/lib") except if the path is of the form "/c/foo" which is treated as "C:/foo" - removes all Windows-specific logic from Path PathFragment is still aware of drive letters and it has to remain so because it is unaware of file systems. WindowsPath restricts the allowed path strings to absolute Unix paths where the first segment, if any, is a volume specifier. From now on if Bazel attempts to create a WindowsPath from an absolute Unix path, Bazel will make it relative to WindowsPath.UNIX_ROOT, unless the first component is a single-letter name (e.g. "/c/foo" which is "C:/foo"). Subclassing Path is necessary because a Unix-style absolute path doesn't sufficiently define a full Windows path, as it may be relative to any drive. Fixes https://github.com/bazelbuild/bazel/issues/1463 -- MOS_MIGRATED_REVID=137149483
* Rollback of commit e0d7a540e3c615c628f63fcaaaba0c47fca2cb25.Gravatar Philipp Wollermann2016-10-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Suspected root cause for Windows bootstrap on Bazel CI breakage: java.lang.NullPointerException at com.google.devtools.build.lib.vfs.Path$1.run(Path.java:123) http://ci.bazel.io/view/Bazel%20bootstrap%20and%20maintenance/job/Bazel/922/JAVA_VERSION=1.8,PLATFORM_NAME=windows-x86_64/console *** Original change description *** VFS: implement a Windows-specific Path subclass The new subclass WindowsFileSystem.WindowsPath is aware of Windows drives. This change: - introduces a new factory for Path objects so FileSystems can return a custom implementation that instantiates filesystem-specific Paths - implements the WindowsPath subclass of Path that is aware of Windows drives - introduces the bazel.windows_unix_root JVM argument that defines the MSYS root, which defines the absolute Windows path that is the... *** -- MOS_MIGRATED_REVID=136583352
* VFS: implement a Windows-specific Path subclassGravatar Laszlo Csomor2016-10-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The new subclass WindowsFileSystem.WindowsPath is aware of Windows drives. This change: - introduces a new factory for Path objects so FileSystems can return a custom implementation that instantiates filesystem-specific Paths - implements the WindowsPath subclass of Path that is aware of Windows drives - introduces the bazel.windows_unix_root JVM argument that defines the MSYS root, which defines the absolute Windows path that is the root of all Unix paths that Bazel creates (e.g. "/usr/lib" -> "C:/tools/msys64/usr/lib") except if the path is of the form "/c/foo" which is treated as "C:/foo" - removes all Windows-specific logic from Path PathFragment is still aware of drive letters and it has to remain so because it is unaware of file systems. WindowsPath restricts the allowed path strings to absolute Unix paths where the first segment, if any, is a volume specifier. From now on if Bazel attempts to create a WindowsPath from an absolute Unix path, Bazel will make it relative to WindowsPath.UNIX_ROOT, unless the first component is a single-letter name (e.g. "/c/foo" which is "C:/foo"). Subclassing Path is necessary because a Unix-style absolute path doesn't sufficiently define a full Windows path, as it may be relative to any drive. Fixes https://github.com/bazelbuild/bazel/issues/1463 -- MOS_MIGRATED_REVID=136350304
* Exporting functions to compute SHA1 digests of files.Gravatar Ola Rozenfeld2016-09-12
| | | | | | | | TODO: need to stich them all the way through various ActionInputFileCache implementations! -- MOS_MIGRATED_REVID=132685408
* Fixed the issue that hard links are handled improperly when bazel ↵Gravatar Googler2016-09-08
| | | | | | | | | decompresses tarballs. Issue link: https://github.com/bazelbuild/bazel/issues/574 -- MOS_MIGRATED_REVID=132434278
* Windows, tests: move code into WindowsTestUtilGravatar Laszlo Csomor2016-09-07
| | | | | | | | | | | | Additionally: - clean up the corresponding BUILD file a bit - add a comment to Path A subsequent change will add tests for WindowsFileSystem, then fix a bug there. -- MOS_MIGRATED_REVID=132408212
* Fix Windows issue in Path.getRelative, which wasn't handling backslashes.Gravatar Ulf Adams2016-05-19
| | | | | -- MOS_MIGRATED_REVID=122627792
* Support case-insensitive comparision in Path.java with WindowsFileSystemGravatar Yun Peng2016-05-09
| | | | | | | | Since file path is case-insensitive on Windows, we need to support this. Also fixed .d file inclusions check in CppCompileAction.java on Windows -- MOS_MIGRATED_REVID=121823250
* Short-circuit Path.equals() with hashcode.Gravatar Googler2016-04-21
| | | | | | | Optimize .equals() by using the hashcode calculated in the constructor to determine if we can short-circuit equals and avoid traversing the parents if the two paths can't be equal. -- MOS_MIGRATED_REVID=120363376
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Introduce Path#isSpecialFile, FileSystem#isSpecialFile, and ↵Gravatar Nathan Harmata2015-10-21
| | | | | | | FileStatus#isSpecialFile to help disambiguate between a regular file and a special file, since the file size of a special file cannot be trusted. -- MOS_MIGRATED_REVID=105903622
* Description redacted.Gravatar Nathan Harmata2015-10-09
| | | | | -- MOS_MIGRATED_REVID=105050723
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Introduce Path#readSymbolicLinkUnchecked, intended to only be used when the ↵Gravatar Nathan Harmata2015-09-17
| | | | | | | caller already knows the path is a symlink, and use this new method throughout the codebase. -- MOS_MIGRATED_REVID=103229983
* Remove unused 'followSymlinks' parameter to FileSystem#getxattr. It is ↵Gravatar Nathan Harmata2015-09-16
| | | | | | | always true in practice. -- MOS_MIGRATED_REVID=103221081
* Rollback of not-actually-thread-safe code introduced in a previous change. ↵Gravatar Nathan Harmata2015-09-03
| | | | | | | IdentityHashMap#get can't be safely called concurrently with #put. -- MOS_MIGRATED_REVID=102189513
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-27 06:46:39 +0000