aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/vfs
Commit message (Collapse)AuthorAge
* Clean up code that directly imports nested classes like Builder, Entry, etc.Gravatar jcater2018-05-01
| | | | PiperOrigin-RevId: 195040539
* Normalize parameter name commentsGravatar cushon2018-04-27
| | | | PiperOrigin-RevId: 194512971
* Make FileSystemUtils.moveFile always preserve symlinks and use it in ↵Gravatar Benjamin Peterson2018-04-12
| | | | | | | | | | | | | | | | | | | | | | | SandboxedSpawn.copyOutputs. Previously, if moveFile fell back to copying from a true rename (e.g., if the move is across file systems), it would fully dereference the source and produce a regular file at the output location. This CL fixes moveFile to properly copy symlinks. Note we don't preserve metadata in the symlink case mostly because the Bazel VFS has no API to write metadata without dereferencing symlinks. (But, also, it's not important for the current use cases.) This breaks the backward-compatibility of FileSystemUtils.moveFile and FileSystemUtils.moveTreeBelow. This seems okay because the new behavior makes more sense, and sandbox is the only consumer of these APIs. Switching SandboxedSpawn.copyOutputs to use FileSystemUtils.moveFile rather than Guava's Files.move fixes https://github.com/bazelbuild/bazel/issues/4987. Closes #4989. Change-Id: I0283e8aa11fadff5b0afd7bdfd0490aca12a1f6b PiperOrigin-RevId: 192611227
* Make some objects frequently encountered during serialization into ↵Gravatar janakr2018-04-10
| | | | | | | | constants. A lot of care is needed here because we're using reference equality. I plan to add value-equality constants in a follow-up. Add ImmutableSortedSet marshaller because I think it might have been needed, and hey, why not. PiperOrigin-RevId: 192326359
* Change profiling to only accept strings for its "description" argument. ↵Gravatar janakr2018-04-01
| | | | | | Profiling can hold onto objects for the duration of the build, and some of those objects may be temporary that should not be persisted. In particular, UnixGlob and its inner classes should not outlive loading and analysis. For the most part, care was taken in this CL to only use strings that required no additional construction, mainly to minimize garbage (retaining references to newly created strings is not as great a concern since only the strings corresponding to the slowest K tasks are retained, for some relatively small values of K). Action descriptions for actually executing actions are eagerly expanded because that work is minimal compared to the work of actually executing an action. PiperOrigin-RevId: 191251488
* Refactor and cleanup the sandboxing code.Gravatar Philipp Wollermann2018-03-23
| | | | | | | | | | | | | - Remove Optional<> where it's not needed. It's nice for return values, but IMHO it was overused in this code (e.g. Optional<List<X>> is an anti-pattern, as the list itself can already signal that it is empty). - Use Bazel's own Path class when dealing with paths, not String or java.io.File. - Move LinuxSandboxUtil into the "sandbox" package. - Remove dead code and unused fields. - Migrate deprecated VFS method calls to their replacements. - Fix a bug in ExecutionStatistics where a FileInputStream was not closed. Closes #4868. PiperOrigin-RevId: 190217476
* Optimize GC churn due to PackageFunction#getContainingDirectory. While I'm ↵Gravatar nharmata2018-03-15
| | | | | | | | | here, also slightly restructure the code in #handleLabelsCrossingSubpackagesAndPropagateInconsistentFilesystemExceptions to make it more readable (and defer PackageIdentifier allocations). Alternatives considered: Add a PathFragment#getParentDirectoryOfRelative(String other) instance method, and use it in PackageFunction#getContainingDirectory. I thought the approach in this CL would be preferable to adding a specialized method like that to PathFragment. RELNOTES: None PiperOrigin-RevId: 189197855
* Add mostly-ide-generated hashCode for RootedPathGravatar michajlo2018-03-12
| | | | | | | Removed null-checks since not expecting null. RootedPath's hashCode is called enough that it's worth optimizing. PiperOrigin-RevId: 188801280
* @AutoCodec TransitiveTraversalValue, DirectoryListingStateValue.Gravatar janakr2018-03-06
| | | | PiperOrigin-RevId: 188077900
* Serialize Path using AutoCodec.Gravatar janakr2018-03-05
| | | | PiperOrigin-RevId: 187945746
* Deletes CODEC fields now that they are no longer needed.Gravatar shahan2018-02-28
| | | | PiperOrigin-RevId: 187397314
* Optimize Path (and the underlying String) allocations in UnixGlob for the ↵Gravatar nharmata2018-02-27
| | | | | | | | | case where a pattern like '*.blah' doesn't match all the dirents. Before the recent Path.java rewrite, the unconditional allocation here was free/amortized (in the sense that if Blaze ever had to consider the path in question, then we'd be doing the Path allocation anyway and caching the object). But now it's potentially GC churny, especially in the case where a single BUILD file has multiple glob expressions. RELNOTES: None PiperOrigin-RevId: 187185573
* Adds ObjectCodecRegistry to {Des|S}erializationContext.Gravatar shahan2018-02-20
| | | | | | | | | | | | * AutoCodec now delegates to the registry. * Adds getSuperclass logic for resolving a codec. * Small cleanups for classes that break the registry. TODO after this change: * Explicit CODEC definitions are no longer needed and existing ones should be cleaned up. * POLYMORPHIC is no longer be needed and should be cleaned up. PiperOrigin-RevId: 186351845
* Move CommandLine, CommandLineItem, and ParamFileInfo from ↵Gravatar tomlu2018-02-15
| | | | | | | | | lib.analysis.actions -> lib.actions. These are fundamental types that want to sit alongside types like Spawn. RELNOTES: None PiperOrigin-RevId: 185887971
* Don't intern PathFragments when we create PackageIdentifiers. This should be ↵Gravatar shreyax2018-02-15
| | | | | | | | a strict win: There are only two places Canonicalizer did PathFragment interning: PackageIdentifier creation and Package creation. PackageIdentifiers are always interned by a separate interner, and so the underlying PathFragment will be in 1-1 correspondence with PackageIdentifiers (per repo). Moreover, Packages are created with a PackageIdentifier already existing, so it will use the same PathFragment of the unique PackageIdentifier. PiperOrigin-RevId: 185877942
* Replaces InjectingObjectCodec with dependencies threaded through ↵Gravatar shahan2018-02-13
| | | | | | (Des|S)erializationContext. PiperOrigin-RevId: 185547740
* Add context argument to ObjectCodec.{serialize,deserialize}Gravatar michajlo2018-02-11
| | | | | | | | | | Context implementations are currently empty, just doing the plumbing in this change. Once this is in we can start passing along the ObjectCodecRegistry, which will allow runtime codec resolution for classes not known at compile time. We'll also inevitably add some memoization helpers, allowing us to optimize the serialization process further. PiperOrigin-RevId: 185305674
* Do not store Path object in RootedPath.Gravatar tomlu2018-02-09
| | | | | | | This can be computed on the fly if we need it. RELNOTES: None PiperOrigin-RevId: 185180257
* Replace path implementation.Gravatar tomlu2018-02-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Path and PathFragment have been replaced with String-based implementations. They are pretty similar, but each method is dissimilar enough that I did not feel sharing code was appropriate. A summary of changes: PATH ==== * Subsumes LocalPath (deleted, its tests repurposed) * Use a simple string to back Path * Path instances are no longer interned; Reference equality will no longer work * Always normalized (same as before) * Some operations will now be slower, like instance compares (which were previously just a reference check) * Multiple identical paths will now consume more memory since they are not interned PATH FRAGMENT ============= * Use a simple string to back PathFragment * No more segment arrays with interned strings * Always normalized * Remove isNormalized * Replace some isNormalizied uses with containsUpLevelReferences() to check if path fragments try to escape their scope * To check if user input is normalized, supply static methods on PathFragment to validate the string before constructing a PathFragment * Because PathFragments are always normalized, we have to replace checks for literal "." from PathFragment#getPathString to PathFragment#getSafePathString. The latter returns "." for the empty string. * The previous implementation supported efficient segment semantics (segment count, iterating over segments). This is now expensive since we do longer have a segment array. ARTIFACT ======== * Remove Path instance. It is instead dynamically constructed on request. This is necessary to avoid this CL becoming a memory regression. RELNOTES: None PiperOrigin-RevId: 185062932
* Remove Root#getCodec, to bring Root in line with general style for codecGravatar cpeyser2018-02-02
| | | | | | declaration. PiperOrigin-RevId: 184304653
* Rephrase RootCodec as an InjectingObjectCodec.Gravatar cpeyser2018-01-29
| | | | PiperOrigin-RevId: 183677348
* Add CommandLineItem interface.Gravatar tomlu2018-01-25
| | | | | | | | This interface makes it clearer in the type system exactly how items that go into a CustomCommandLine are turned into strings. It is a preparatory change to allow command line fingerprints to be more cheaply calculated, but it is valuable in itself from a code quality standpoint. PiperOrigin-RevId: 183274022
* Fix usages of PathFragment segments that will become inefficient.Gravatar tomlu2018-01-19
| | | | | | | An upcoming replacement to PathFragment will not have efficient segment semantics, causing code to become unnecessarily inefficient. RELNOTES: None PiperOrigin-RevId: 182553098
* Rename relativePath -> rootRelativePath in Root and friends.Gravatar tomlu2018-01-19
| | | | | | This makes it clearer that the path fragments in question are relative *to the root*. Confusingly, when the root is absolute, the root relative fragment is also absolute. This makes it a tiny bit clearer that the path fragment may be absolute. PiperOrigin-RevId: 182544893
* Inline rarely used PathFragment methods that do not deserve to be on a core ↵Gravatar tomlu2018-01-19
| | | | | | path class. PiperOrigin-RevId: 182526427
* Address post-commit comments in ↵Gravatar nharmata2018-01-18
| | | | | | | https://github.com/bazelbuild/bazel/commit/6f35e2d81a4d10359d39643bd03887d1b87f4224. RELNOTES: None PiperOrigin-RevId: 182415982
* Add absolute root concept.Gravatar tomlu2018-01-18
| | | | | | | | An absolute root accepts any absolute path fragments and simply returns it. This concept replaces the FileSystem root directory concept. PiperOrigin-RevId: 182400471
* Introduce Root class.Gravatar tomlu2018-01-17
| | | | | | | | | | | This class represents a root (such as a package path or an output root) used for file lookups and artifacts. It is meant to be as opaque as possible in order to hide the user's environment from sky keys and sky functions. Roots are used by RootedPaths and ArtifactRoots. This CL attempts to make the minimum number of modifications necessary to change RootedPath and ArtifactRoot to use these fields. Deprecated methods and invasive accessors are permitted to minimise the risk of any observable changes. RELNOTES: None PiperOrigin-RevId: 182271759
* Have UnixGlob explicitly handle RuntimeExceptions encountered by individual ↵Gravatar nharmata2018-01-16
| | | | | | | glob tasks. RELNOTES: None PiperOrigin-RevId: 182125500
* Move RootedPathCodec to an inner class of RootedPath.Gravatar tomlu2018-01-16
| | | | PiperOrigin-RevId: 182087153
* Adds FileSystem check to Path.CODEC.serialize.Gravatar shahan2018-01-12
| | | | PiperOrigin-RevId: 181797078
* Change packaging rules to operate on strings instead of path fragments.Gravatar tomlu2018-01-12
| | | | | | | | | | | | The upcoming path refactor will normalize all path fragments upon creation. That is fine 99% of the time, but sometimes we want to disallow non-normalized paths on rule attributes. Even this isn't usually a problem since we can validate the string prior to putting it in a path fragment. However, in the case of the packaging rule we do not know the rules of validation until the packaging rule is *consumed* by some other rule. Therefore, retain input as a string on these rules all the way through. We don't really do a lot of path fragmenty stuff with these strings. The main drawback is losing a bit of type safety / readability. SKIP_KOKORO PiperOrigin-RevId: 181760613
* Factor out InMemoryFileSystem's use of root directory.Gravatar tomlu2018-01-11
| | | | | | When there is no root directory this minimises the size of the change required to keep InMemoryFileSystem working. PiperOrigin-RevId: 181685159
* Adds a CODEC for CppConfiguration.Gravatar shahan2018-01-10
| | | | | | | * Creates an enum for cpu transformer, which is easier to serialize than an opaque function. This also means moving FakeCPU to avoid introducing a circular dependency. * Adds a CODEC to Path using InjectingObjectCodec. PiperOrigin-RevId: 181445911
* Automated rollback of commit 80c8ff1819a889814cbe2dae477d7fedce6aa181.Gravatar tomlu2018-01-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Rolling forward after underlying issue has been fixed. *** Original change description *** Automated rollback of commit d50cbbeef115f28c0cea1ac17572e0f12c0cf312. *** Reason for rollback *** b/71442447 *** Original change description *** Remove synchronization from file system. After the path refactor we will no longer have path instances to synchronize on. The underlying OS file systems are already naturally thread safe, that is, their internal data structures cannot be damaged. Any further synchronization (eg. races between directory creation and deletion) has to be managed at the client level. The l... *** PiperOrigin-RevId: 181368707
* Automated rollback of commit cb7689404ef9a69488d64bfc2e0bfb9326e664d6.Gravatar tomlu2018-01-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Rolling forward after the underlying issue has been fixed. *** Original change description *** Automated rollback of commit 6a54339bb943702bd7dffc43f85267dac98dc355. *** Reason for rollback *** b/71442447 *** Original change description *** Call through to Path#createDirectoryAndParents from FileUtils. This CL removes a method that due to its implementation causes threading difficulties for Path#createDirectory. The tests for the method are brought across to FileSystemTests since the methods are now implemented natively by the FileSystem classes. The tests were also cleaned up. The test revealed an edge c... *** PiperOrigin-RevId: 181367850
* Automated rollback of commit 6a54339bb943702bd7dffc43f85267dac98dc355.Gravatar tomlu2018-01-05
| | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** b/71442447 *** Original change description *** Call through to Path#createDirectoryAndParents from FileUtils. This CL removes a method that due to its implementation causes threading difficulties for Path#createDirectory. The tests for the method are brought across to FileSystemTests since the methods are now implemented natively by the FileSystem classes. The tests were also cleaned up. The test revealed an edge case bug in JavaIoFileSystem, so fix this. In two cases some code was using the return value from the old method. Returning "f... *** ROLLBACK_OF=179864042 PiperOrigin-RevId: 180946251
* Automated rollback of commit d50cbbeef115f28c0cea1ac17572e0f12c0cf312.Gravatar tomlu2018-01-05
| | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** b/71442447 *** Original change description *** Remove synchronization from file system. After the path refactor we will no longer have path instances to synchronize on. The underlying OS file systems are already naturally thread safe, that is, their internal data structures cannot be damaged. Any further synchronization (eg. races between directory creation and deletion) has to be managed at the client level. The last attempt to do this failed because of races in FileUtils#createDirectoryAndParents on Windows. This method is now gone, rep... *** ROLLBACK_OF=180290901 PiperOrigin-RevId: 180936132
* Remove support for Windows relative-to-current-drive paths.Gravatar tomlu2018-01-05
| | | | | | Eg. 'C:foo' was previously "the directory 'foo' relative to the current directory of drive 'C:\'". Now it is simple interpreted as "the relative path fragment 'C:foo'". PiperOrigin-RevId: 180936012
* Remove synchronization from file system.Gravatar tomlu2017-12-28
| | | | | | | | | | After the path refactor we will no longer have path instances to synchronize on. The underlying OS file systems are already naturally thread safe, that is, their internal data structures cannot be damaged. Any further synchronization (eg. races between directory creation and deletion) has to be managed at the client level. The last attempt to do this failed because of races in FileUtils#createDirectoryAndParents on Windows. This method is now gone, replaced by a method from the Java framework that knows how to synchronize. PiperOrigin-RevId: 180290901
* Undo change to introduce methods to remove file system instance from Path.Gravatar tomlu2017-12-28
| | | | | | | | I no longer believe that it is worthwhile to remove the file system instance from Path. Measurements indicate that I can simply replace the Path implementation in one go and just leave the instance on the Path class. This CL was done entirely using IDE refactoring, so should be safe. PiperOrigin-RevId: 180287162
* Redo FileType to reduce generated garbage.Gravatar tomlu2017-12-22
| | | | | | | | | | | | | * Change FileType to no longer assume it operates on just the base name (it can now be given a full path). * Move the responsibility to specific classes (Artifact, Path, PathFragment) to decide how they want to offer up a string that includes the file name. * Flip the order in which users are expected to check Artifact type, from FileType#matches(Artifact) to Artifact#isFileType(FileType). This looks natural and should encourage developers to use efficient file type checking methods. * Change CppCompileAction to use the new API. RELNOTES: None PiperOrigin-RevId: 179903239
* Call through to Path#createDirectoryAndParents from FileUtils.Gravatar tomlu2017-12-21
| | | | | | | | | | | | This CL removes a method that due to its implementation causes threading difficulties for Path#createDirectory. The tests for the method are brought across to FileSystemTests since the methods are now implemented natively by the FileSystem classes. The tests were also cleaned up. The test revealed an edge case bug in JavaIoFileSystem, so fix this. In two cases some code was using the return value from the old method. Returning "false" essentially means that the directory already existed, and the code doesn't look racy, so we can replace it with an existence check. PiperOrigin-RevId: 179864042
* Remove cached version of FileSystemUtils#createDirectoryAndParents.Gravatar tomlu2017-12-21
| | | | | | Only the sandbox uses the cached version. We can let it do its own caching of the top-level directory. It won't help with higher-level directories, but if there are a lot of inputs in the same directory we'll avoid I/O there. PiperOrigin-RevId: 179830651
* Add FileSystem#createDirectoryAndParents.Gravatar tomlu2017-12-21
| | | | | | | | A native implementation of this (instead of using FileSystemUtils, which can only use public interfaces) should be more efficient and more easy to make correct. In particular, it should allow removing FileSystemUtils#createDirectoriesAndParents, which has poor thread safety characteristics. The latter method has a lot of logic that forces certain unnatural atomicity guarantees on createDirectory, and it also has logic that is conditional on sub-string content of exception messages. PiperOrigin-RevId: 179819623
* Correct typos found by the linter.Gravatar gregce2017-12-20
| | | | PiperOrigin-RevId: 179748374
* Makes PathFragment have internal CODEC definition.Gravatar Googler2017-12-19
| | | | PiperOrigin-RevId: 179588512
* Automated rollback of commit 82e68b75304438c96ff878a0c2b8d18b42002486.Gravatar aehlig2017-12-19
| | | | | | | | | | | | | | Fixes #4322, #4306. *** Reason for rollback *** Introduces a deadlock (see https://github.com/bazelbuild/bazel/issues/4322) *** Original change description *** Make FileSystem operate on LocalPath instead of Path. PiperOrigin-RevId: 179549866
* Make FileSystem operate on LocalPath instead of Path.Gravatar tomlu2017-12-14
| | | | PiperOrigin-RevId: 179082062
* Move msys path support into DependencySet, removing it from the general Path ↵Gravatar tomlu2017-12-14
| | | | | | | | system. This is the only place that should actually need it. PiperOrigin-RevId: 179054861
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-07 00:17:52 +0000