| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Go back to the default constructor - instead of requiring everywhere to know the correct hash function, we'll have the default rely on global state. It will make transition easier, even if it makes the origin of the hash less obvious.
*** Original change description ***
Remove default MD5 in most of Bazel's virtual filesystems.
This forces the ex-default to be explicit in a lot of tests, but I'd rather that than have the risk of implicit md5-use in production code.
To keep this CL smaller, do not remove the default from UnixFS quite yet.
RELNOTES: None.
PiperOrigin-RevId: 206358838
|
|
|
|
|
|
|
|
|
| |
This forces the ex-default to be explicit in a lot of tests, but I'd rather that than have the risk of implicit md5-use in production code.
To keep this CL smaller, do not remove the default from UnixFS quite yet.
RELNOTES: None.
PiperOrigin-RevId: 206223521
|
|
|
|
|
|
|
| |
Each FileSystem instance has a digest function, but the getDigest and getHashDigest functions also accepted their own custom parameter functions. We only support 1 hash per filesystem instance, these parameters are redundant.
RELNOTES: None.
PiperOrigin-RevId: 205758571
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
JavaIoFileSystem.delete() now uses
Files.deleteIfExists() to delete files and empty
directories.
The old code used to have a race condition: a
file could have been deleted or changed to a
non-directory between checking it's a directory
and listing its contents.
WindowsFileSystem.delete() now uses the
DeletePath JNI method (wrapped by
WindowsFileOperations.deletePath) which is more
robust than JavaIoFileSystem.delete(), because it
can delete readonly files and can tolerate some
concurrent filesystem modifications.
Fixes https://github.com/bazelbuild/bazel/issues/5513
Change-Id: I5d2720afff8dfd3725880a6d7408d22de5935c3d
Closes #5527.
Change-Id: I5d2720afff8dfd3725880a6d7408d22de5935c3d
PiperOrigin-RevId: 203720575
|
|
|
|
|
|
|
|
|
| |
enum.
Now that we aren't using enum names for the hash functions, we also accept the standard names, such as SHA-256.
RELNOTES: None.
PiperOrigin-RevId: 201624286
|
|
|
|
|
|
| |
Profiling can hold onto objects for the duration of the build, and some of those objects may be temporary that should not be persisted. In particular, UnixGlob and its inner classes should not outlive loading and analysis. For the most part, care was taken in this CL to only use strings that required no additional construction, mainly to minimize garbage (retaining references to newly created strings is not as great a concern since only the strings corresponding to the slowest K tasks are retained, for some relatively small values of K). Action descriptions for actually executing actions are eagerly expanded because that work is minimal compared to the work of actually executing an action.
PiperOrigin-RevId: 191251488
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Path and PathFragment have been replaced with String-based implementations. They are pretty similar, but each method is dissimilar enough that I did not feel sharing code was appropriate.
A summary of changes:
PATH
====
* Subsumes LocalPath (deleted, its tests repurposed)
* Use a simple string to back Path
* Path instances are no longer interned; Reference equality will no longer work
* Always normalized (same as before)
* Some operations will now be slower, like instance compares (which were previously just a reference check)
* Multiple identical paths will now consume more memory since they are not interned
PATH FRAGMENT
=============
* Use a simple string to back PathFragment
* No more segment arrays with interned strings
* Always normalized
* Remove isNormalized
* Replace some isNormalizied uses with containsUpLevelReferences() to check if path fragments try to escape their scope
* To check if user input is normalized, supply static methods on PathFragment to validate the string before constructing a PathFragment
* Because PathFragments are always normalized, we have to replace checks for literal "." from PathFragment#getPathString to PathFragment#getSafePathString. The latter returns "." for the empty string.
* The previous implementation supported efficient segment semantics (segment count, iterating over segments). This is now expensive since we do longer have a segment array.
ARTIFACT
========
* Remove Path instance. It is instead dynamically constructed on request. This is necessary to avoid this CL becoming a memory regression.
RELNOTES: None
PiperOrigin-RevId: 185062932
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Rolling forward after underlying issue has been fixed.
*** Original change description ***
Automated rollback of commit d50cbbeef115f28c0cea1ac17572e0f12c0cf312.
*** Reason for rollback ***
b/71442447
*** Original change description ***
Remove synchronization from file system.
After the path refactor we will no longer have path instances to synchronize on.
The underlying OS file systems are already naturally thread safe, that is, their internal data structures cannot be damaged. Any further synchronization (eg. races between directory creation and deletion) has to be managed at the client level.
The l...
***
PiperOrigin-RevId: 181368707
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Rolling forward after the underlying issue has been fixed.
*** Original change description ***
Automated rollback of commit 6a54339bb943702bd7dffc43f85267dac98dc355.
*** Reason for rollback ***
b/71442447
*** Original change description ***
Call through to Path#createDirectoryAndParents from FileUtils.
This CL removes a method that due to its implementation causes threading difficulties for Path#createDirectory.
The tests for the method are brought across to FileSystemTests since the methods are now implemented natively by the FileSystem classes. The tests were also cleaned up.
The test revealed an edge c...
***
PiperOrigin-RevId: 181367850
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
b/71442447
*** Original change description ***
Call through to Path#createDirectoryAndParents from FileUtils.
This CL removes a method that due to its implementation causes threading difficulties for Path#createDirectory.
The tests for the method are brought across to FileSystemTests since the methods are now implemented natively by the FileSystem classes. The tests were also cleaned up.
The test revealed an edge case bug in JavaIoFileSystem, so fix this.
In two cases some code was using the return value from the old method. Returning "f...
***
ROLLBACK_OF=179864042
PiperOrigin-RevId: 180946251
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
b/71442447
*** Original change description ***
Remove synchronization from file system.
After the path refactor we will no longer have path instances to synchronize on.
The underlying OS file systems are already naturally thread safe, that is, their internal data structures cannot be damaged. Any further synchronization (eg. races between directory creation and deletion) has to be managed at the client level.
The last attempt to do this failed because of races in FileUtils#createDirectoryAndParents on Windows. This method is now gone, rep...
***
ROLLBACK_OF=180290901
PiperOrigin-RevId: 180936132
|
|
|
|
|
|
|
|
|
|
| |
After the path refactor we will no longer have path instances to synchronize on.
The underlying OS file systems are already naturally thread safe, that is, their internal data structures cannot be damaged. Any further synchronization (eg. races between directory creation and deletion) has to be managed at the client level.
The last attempt to do this failed because of races in FileUtils#createDirectoryAndParents on Windows. This method is now gone, replaced by a method from the Java framework that knows how to synchronize.
PiperOrigin-RevId: 180290901
|
|
|
|
|
|
|
|
|
|
|
|
| |
This CL removes a method that due to its implementation causes threading difficulties for Path#createDirectory.
The tests for the method are brought across to FileSystemTests since the methods are now implemented natively by the FileSystem classes. The tests were also cleaned up.
The test revealed an edge case bug in JavaIoFileSystem, so fix this.
In two cases some code was using the return value from the old method. Returning "false" essentially means that the directory already existed, and the code doesn't look racy, so we can replace it with an existence check.
PiperOrigin-RevId: 179864042
|
|
|
|
|
|
|
|
| |
A native implementation of this (instead of using FileSystemUtils, which can only use public interfaces) should be more efficient and more easy to make correct.
In particular, it should allow removing FileSystemUtils#createDirectoriesAndParents, which has poor thread safety characteristics. The latter method has a lot of logic that forces certain unnatural atomicity guarantees on createDirectory, and it also has logic that is conditional on sub-string content of exception messages.
PiperOrigin-RevId: 179819623
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes #4322, #4306.
*** Reason for rollback ***
Introduces a deadlock (see https://github.com/bazelbuild/bazel/issues/4322)
*** Original change description ***
Make FileSystem operate on LocalPath instead of Path.
PiperOrigin-RevId: 179549866
|
|
|
|
| |
PiperOrigin-RevId: 179082062
|
|
|
|
| |
PiperOrigin-RevId: 178704585
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 178426166
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Refactor the FileSystem class to include the hash function as an
instance field. This allows us to have a different hash function
per FileSystem and removes technical debt, as currently that's
somewhat accomplished by a horrible hack that has a static method
to set the hash function for all FileSystem instances.
The FileSystem's default hash function remains MD5.
RELNOTES: None
PiperOrigin-RevId: 177479772
|
|
|
|
|
|
|
|
|
|
|
| |
As explained in the javadoc this avoids unnecessary allocations and
preserves underlying Java filesystem information.
This partially addresses #3952.
Closes #3953.
PiperOrigin-RevId: 173877045
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Break CI Windows nightly build: https://github.com/bazelbuild/bazel/issues/3927
https://ci.bazel.io/blue/organizations/jenkins/Global%2Fbazel-tests/detail/bazel-tests/314/pipeline/31
*** Original change description ***
Remove synchronization from FileSystem implementations.
PiperOrigin-RevId: 173243523
|
|
|
|
|
|
|
|
| |
file/directory names instead of paths.
This is a small isolated change that can be done ahead of the big refactoring.
PiperOrigin-RevId: 173124518
|
|
|
|
|
|
|
|
| |
This works better with UnionFileSystem, as (eg.) different mapped file systems might not agree on whether they are read-only.
No actual code changes have been made that actually uses this argument, so this should be a behaviour no-op.
PiperOrigin-RevId: 172782759
|
|
|
|
| |
PiperOrigin-RevId: 171905267
|
|
|
|
|
|
|
|
|
| |
Split collect, concurrent, vfs, windows into package-level BUILD files.
Move clock classes out of "util", into their own Java package.
Move CompactHashSet into its own Java package to break a dependency cycle.
Give nestedset and inmemoryfs their own package-level BUILD files.
PiperOrigin-RevId: 167702127
|
|
|
|
|
|
|
|
|
| |
the currently defined hash function for blobs. Some refactoring. Adding an option to set the hash function in the remote worker, defaulting to the current behavior (unfortunately it is a build option, have not found a clean way to specify it at runtime).
BUG=62622420
TESTED=remote worker
RELNOTES: none
PiperOrigin-RevId: 159473116
|
|
|
|
|
|
|
|
|
|
|
|
| |
'create' method.
This paves the way for changing PathFragment to e.g. an abstract class with multiple subclasses. This way we can split out the windows-specific stuff into one of these concrete classes, making the code more readable and also saving memory (since the shallow heap size of the NonWindowsPathFragment subclass will hopefully be smaller than that of the current PathFragment).
This also lets us pursue gc churn optimizations. We can now do interning in PathFragment#create and can also get rid of unnecessary intermediate PathFragment allocations.
RELNOTES: None
PiperOrigin-RevId: 152145768
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 148749485
MOS_MIGRATED_REVID=148749485
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It was previously in unix, but also used from non-unix file systems, which
means it's not actually unix-specific. This is in preparation for splitting
compilation of the unix and windows file systems into separate libraries.
That improves layering and reduces compile times - note that Bazel already
injects the vfs into its lower layers, which should only rely on the normal
vfs APIs, not on anything platform-specific.
--
PiperOrigin-RevId: 147829659
MOS_MIGRATED_REVID=147829659
|
|
|
|
|
|
|
|
|
| |
decompresses tarballs.
Issue link: https://github.com/bazelbuild/bazel/issues/574
--
MOS_MIGRATED_REVID=132434278
|
|
|
|
|
|
|
| |
behavior (passing -1 modified time should use system time).
--
MOS_MIGRATED_REVID=128489592
|
|
|
|
|
|
|
|
| |
Since file path is case-insensitive on Windows, we need to support this.
Also fixed .d file inclusions check in CppCompileAction.java on Windows
--
MOS_MIGRATED_REVID=121823250
|
|
|
|
|
|
|
|
|
| |
changes between his branch and HEAD.
This code will need some JNI to make it work properly, but for now, it will do.
--
MOS_MIGRATED_REVID=113349143
|
|
|
|
|
|
|
| |
our plans for symlink support on Windows.
--
MOS_MIGRATED_REVID=113043269
|
|
|
|
|
|
|
|
|
| |
(accidentally) the implicit requirement that the deriving class override FileSystem#stat.
Even though this wasn't a problem in practice in the Bazel codebase (since all of our transitive subclasses had custom 'stat' implementation), it's good hygiene to have things correct (e.g. if we added a new subclass).
--
MOS_MIGRATED_REVID=108158039
|
|
|
|
|
|
|
| |
FileStatus#isSpecialFile to help disambiguate between a regular file and a special file, since the file size of a special file cannot be trusted.
--
MOS_MIGRATED_REVID=105903622
|
|
|
|
|
|
|
|
|
|
|
| |
The headers were modified with
`find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'`
And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan.
The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand.
--
MOS_MIGRATED_REVID=103938715
|
|
|
|
|
|
|
| |
link system call.
--
MOS_MIGRATED_REVID=88441395
|
|
--
MOE_MIGRATED_REVID=85702957
|