| Commit message (Collapse) | Author | Age |
|
|
|
|
|
| |
This requires a fairly large amount of changes to fundamental objects like BlazeRuntime, Executor, and so on, as well as changing a lot of test code to thread the file system through. I expect future CLs to be much smaller.
PiperOrigin-RevId: 173678144
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Break CI Windows nightly build: https://github.com/bazelbuild/bazel/issues/3927
https://ci.bazel.io/blue/organizations/jenkins/Global%2Fbazel-tests/detail/bazel-tests/314/pipeline/31
*** Original change description ***
Remove synchronization from FileSystem implementations.
PiperOrigin-RevId: 173243523
|
|
|
|
|
|
|
|
| |
file/directory names instead of paths.
This is a small isolated change that can be done ahead of the big refactoring.
PiperOrigin-RevId: 173124518
|
|
|
|
|
|
| |
It now correctly delegates to the right file system.
PiperOrigin-RevId: 172916854
|
|
|
|
|
|
| |
Path construction entails calling isFileSystemCaseSensitive at a time when subclasses have not been initialised. This can cause a crash. Solve by deferring init of the path.
PiperOrigin-RevId: 172914501
|
|
|
|
|
|
|
|
| |
Nobody sets a delegate anymore, meaning it should be fine to crash hard if we try to resolve a file outside scope.
The CL was created by inlining ScopeEscapableFileSystem into InMemoryFileSystem, making it impossible to return an outOfScope file status (instead throwing an exception), then trivially fixing the code.
PiperOrigin-RevId: 172886769
|
|
|
|
|
|
| |
This allows symlinks between multiple different file systems. InMemoryFileSystem uses this in some tests. This CL will allow deletion of ScopeEscapableFileSystem, one of two file systems that partially duplicate what UnionFileSystem does.
PiperOrigin-RevId: 172823391
|
|
|
|
|
|
|
|
| |
This works better with UnionFileSystem, as (eg.) different mapped file systems might not agree on whether they are read-only.
No actual code changes have been made that actually uses this argument, so this should be a behaviour no-op.
PiperOrigin-RevId: 172782759
|
|
|
|
| |
PiperOrigin-RevId: 171905267
|
|
|
|
|
|
| |
the tool setup we do in tests doesn't necessarily have to be copied if we copy a workspace over.
PiperOrigin-RevId: 171864170
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In this commit:
* buildenv.sh: restore its state to that as of
commit 511c35b46cead500d4e76706e0a709e50995ceba
* CommonCommandOptions.java: remove a deprecated
no-op flag
* WindowsPathFragment.java: implement an
ASCII-only isLetter function, use that instead
of Character.isLetter, because the latter
returns true for some Unicode characters too
* bazel_bootstrap_distfile_test: remove logging
that we no longer need, since the bugfix for
issue #3618 will be pushed to GitHub later today
Change-Id: Ibda70219e974f0c47bc82addc647d8951f4bd701
PiperOrigin-RevId: 171498977
|
|
|
|
| |
PiperOrigin-RevId: 171221156
|
|
|
|
|
|
|
|
|
| |
Instead of relying on a character-by-character StringTrie, segment paths based on PathFragments. This means UnionFS can accept any path that Bazel stores internally, removing the ASCII limitations.
This also means removing the ability to have a filesystem bound for sub-PathFragments, /foo/barbar, /foo/barqux could have the same filesystem bound at /foo/bar. This feature was tested for when a use case was envisioned, but it was never used, so removing it is safe.
RELNOTES: None.
PiperOrigin-RevId: 170054656
|
|
|
|
|
|
| |
RELNOTES: The Build Event Protocol's File.uri field is now properly
encoded according to RFC2396.
PiperOrigin-RevId: 168674865
|
|
|
|
|
|
|
|
|
| |
Split collect, concurrent, vfs, windows into package-level BUILD files.
Move clock classes out of "util", into their own Java package.
Move CompactHashSet into its own Java package to break a dependency cycle.
Give nestedset and inmemoryfs their own package-level BUILD files.
PiperOrigin-RevId: 167702127
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 167574104
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 167505493
|
|
|
|
|
|
|
|
|
|
|
| |
finished extracting the FDO profile.
Also fix a truly embarrassing infinite recursion bug introduced by Yours Truly in unknown commit .
This avoids a failure mode where, when two profiles at the same path are used in two builds close one after the other, the file handle would get erroneously re-used.
RELNOTES: None.
PiperOrigin-RevId: 163063976
|
|
|
|
|
|
| |
to indicate unserializability, improve error message in Path when serialization fails, and add some test-only methods to SkyframeExecutor and PackageFactory.
PiperOrigin-RevId: 162993806
|
|
|
|
|
|
|
| |
We previously only add those under genfiles root. But J2ObjC and objc proto headers are under the bin root, which we may also fix in the near future.
RELNOTES: None.
PiperOrigin-RevId: 162656166
|
|
|
|
|
|
|
| |
PathFragment.TO_PATH_FRAGMENT
RELNOTES: None.
PiperOrigin-RevId: 160668541
|
|
|
|
|
|
|
| |
With a few manual fixes for readability.
RELNOTES: None.
PiperOrigin-RevId: 160582556
|
|
|
|
|
|
|
|
|
|
|
|
| |
PR #2679 allowed parentheses ("(" and ")") in globs, but other bracket
style characters were not allowed at the time. It was decided in
issues #2583 and #3048 that other bracket characters should be allowed
as well, since there is no apparent historical reason for disallowing
them in the first place.
Closes #3166.
PiperOrigin-RevId: 159691498
|
|
|
|
|
|
|
|
|
| |
the currently defined hash function for blobs. Some refactoring. Adding an option to set the hash function in the remote worker, defaulting to the current behavior (unfortunately it is a build option, have not found a clean way to specify it at runtime).
BUG=62622420
TESTED=remote worker
RELNOTES: none
PiperOrigin-RevId: 159473116
|
|
|
|
|
|
|
| |
9eea05d068a06ab642dd9d86d46ee5fa2e36b02e.
RELNOTES: n/a
PiperOrigin-RevId: 158988688
|
|
|
|
| |
PiperOrigin-RevId: 157218175
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 154543320
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are now four concrete implementations: RelativeUnixPathFragment, AbsoluteUnixPathFragment, RelativeWindowsPathFragment, AbsoluteWindowsPathFragment.
Goals:
-Reduce memory usage of PathFragment on non-Windows platforms while maintaining existing semantics.
-Make a few simple performance improvements along the way.
-Add TODOs for a few more simple performance improvements.
-Open the way for reducing code complexity of PathFragment. All of the Windows-specific stuff ought not complicate the code so much.
Non goals:
-Make the entire codebase as pretty as possible wrt PathFragment & Windows.
-Make PathFragment usage more sane in general (e.g. change semantics to ban coexistence of Windows and Unix PathFragments).
-Optimize PathFragment as much as possible wrt memory or even in any other dimensions (e.g. gc churn, cpu).
To elaborate, the primary motivation is per-instance memory usage of PathFragment on Unix platforms:
Before this change
------------------
+UseCompressedOops --> 32 bytes per instance
-UseCompressedOops --> 40 bytes per instance
After this change
------------------
+UseCompressedOops --> 24 bytes per instance
-UseCompressedOops --> 32 bytes per instance
Since Bazel can retain lots of PathFragments, the memory savings of this CL are fairly large.
RELNOTES: None
PiperOrigin-RevId: 154052905
|
|
|
|
|
|
|
|
|
| |
latter swallows all filesystem failures, and does not disambiguate missing files from filesystem problems.
The syscall cache now tracks IOExceptions if they are present, just as it does with readdir().
RELNOTES: None
PiperOrigin-RevId: 153185433
|
|
|
|
|
|
|
| |
More info here #2583
Closes #2679.
PiperOrigin-RevId: 152685327
|
|
|
|
| |
PiperOrigin-RevId: 152684266
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 152400979
|
|
|
|
|
|
|
|
|
|
|
|
| |
'create' method.
This paves the way for changing PathFragment to e.g. an abstract class with multiple subclasses. This way we can split out the windows-specific stuff into one of these concrete classes, making the code more readable and also saving memory (since the shallow heap size of the NonWindowsPathFragment subclass will hopefully be smaller than that of the current PathFragment).
This also lets us pursue gc churn optimizations. We can now do interning in PathFragment#create and can also get rid of unnecessary intermediate PathFragment allocations.
RELNOTES: None
PiperOrigin-RevId: 152145768
|
|
|
|
|
|
|
|
| |
allocating a garbage TranslatedPath object.
RELNOTES: None
PiperOrigin-RevId: 152130170
|
|
|
|
|
|
|
|
|
|
| |
This fixes PackagePathLocatorTest.
--
Change-Id: I3d14a80993f6b256acfc732adf2d97b1d2dcb804
Reviewed-on: https://cr.bazel.build/9310
PiperOrigin-RevId: 149634730
MOS_MIGRATED_REVID=149634730
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
PathFragment's `equals`, `hashCode`, `compareTo`,
`startsWith`, `endsWith`, and `relativeTo` are now
aware of case-insensitivity when running on
Windows.
This approach is better than
https://bazel-review.googlesource.com/c/9124/
because it preserves path casing, which is
important when computing action output paths.
This change contains two additional bugfixes:
- `compareTo` now takes `driveLetter` into account
- the `InMemoryFileSystem` in `PathWindowsTest` is
not case-insensitive
Fixes https://github.com/bazelbuild/bazel/issues/2613
--
Change-Id: I1a4250a373fff03fa02a6d8360457450b47a42a8
Reviewed-on: https://cr.bazel.build/9126
PiperOrigin-RevId: 149106930
MOS_MIGRATED_REVID=149106930
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 148749485
MOS_MIGRATED_REVID=148749485
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 148625618
MOS_MIGRATED_REVID=148625618
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It was previously in unix, but also used from non-unix file systems, which
means it's not actually unix-specific. This is in preparation for splitting
compilation of the unix and windows file systems into separate libraries.
That improves layering and reduces compile times - note that Bazel already
injects the vfs into its lower layers, which should only rely on the normal
vfs APIs, not on anything platform-specific.
--
PiperOrigin-RevId: 147829659
MOS_MIGRATED_REVID=147829659
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use the new CreateJunction in the Windows JNI code
every time we need to create junctions. This means
updating WindowsFileOperations and related tests.
Add test for WindowsFileSystem.createSymbolicLink.
See https://github.com/bazelbuild/bazel/issues/2238
--
Change-Id: I5827e2e70e8e147f5f102fabf95fa9a148b3bcdc
Reviewed-on: https://cr.bazel.build/8896
PiperOrigin-RevId: 147598107
MOS_MIGRATED_REVID=147598107
|
|
|
|
|
|
|
|
|
|
| |
Fixed https://github.com/bazelbuild/bazel/issues/2494
--
Change-Id: I2ca335fa5b3a7759f57085717128912f09363789
Reviewed-on: https://cr.bazel.build/8650
PiperOrigin-RevId: 146761747
MOS_MIGRATED_REVID=146761747
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Every double-checked lock will be followed up by a regular lock.
Either the children don't exist, in which case we take the lock
twice, or they do exist and we take the lock once.
This makes it so we only ever take the lock once.
I don't know if this makes it faster, but it makes the code
simpler, and shouldn't make it slower.
--
PiperOrigin-RevId: 146553235
MOS_MIGRATED_REVID=146553235
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
TL;DR: resolve "C:/progra~1" style paths, don't
cache failed resolutions in the parent's
Path.children
When creating WindowsPath objects, resolve the
"C:/progra~1" style paths to "C:/program files".
This enables us to correctly compare paths on
Windows. Without this canonicalization we
incorrectly determine "C:/progra~1" and
"C:/program files" to be different when they are
actually the same.
We only attempt to resolve such paths if the name
looks like it's an abbreviated path.
If resolution fails, probably due to the path not
existing, then we don't cache this Path object in
the parent's `children` list. This avoids stale
cache entries in case the path springs into
existence later in the server's lifetime.
Fixes https://github.com/bazelbuild/bazel/issues/2145
We also need to rectify https://github.com/bazelbuild/bazel/issues/2173
--
PiperOrigin-RevId: 143442134
MOS_MIGRATED_REVID=143442134
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Also build the JNI library while bootstraping.
This was once submitted in commit 4a249b6962d32ed4cfd4165dfdae4a555b00bb69 but got
rolled back due to some test breakage that's long
since fixed. In this change I'm slightly modifying
the original code in compile.sh.
Using JNI methods however is necessary because we
can't implement WindowsFileOperations.GetLongPath
in native Java, and having that code is a
prerequisite for the fix of https://github.com/bazelbuild/bazel/issues/2145
See also https://github.com/bazelbuild/bazel/issues/2238
--
PiperOrigin-RevId: 142535019
MOS_MIGRATED_REVID=142535019
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This method converts MSYS paths to Windows path.
It uses the BAZEL_SH envvar to obtain the MSYS
root directory, to which all Unix paths (except
for mounts) are relative.
We cannot handle mounts because we don't want to
read /etc/mtab every time there's a file operation
so we simply apply a heuristic similar to
https://github.com/bazelbuild/bazel/blob/cd4cc09fa6ef96380a3d0888f825dfd1dbada651/src/main/java/com/google/devtools/build/lib/vfs/WindowsFileSystem.java#L52-L63
Also clean up the #ifdefs surrounding SyncFile.
See https://github.com/bazelbuild/bazel/issues/2107
--
PiperOrigin-RevId: 142531986
MOS_MIGRATED_REVID=142531986
|
|
|
|
|
|
|
|
| |
Fixes #2247.
--
PiperOrigin-RevId: 142143520
MOS_MIGRATED_REVID=142143520
|
|
|
|
|
|
|
|
| |
optional method in FileSystem. Custom FileSystem implementations can
use this to provide their own implementation of glob prefetching.
--
MOS_MIGRATED_REVID=140736304
|
|
|
|
|
|
|
|
|
|
|
| |
that makes an appropriate call to Interners.InternerBuilder#concurrencyLevel.
For current readers of this CL, I used this class everywhere in the Blaze codebase.
For future readers of this CL, this class should be used to create an Interner in the Blaze codebase.
--
MOS_MIGRATED_REVID=140063271
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
refactoring: enabling potential fast digest computation of more than one digest function type.
Usage: bazel --host_jvm_args="-Dbazel.DigestFunction=SHA1" build ...
Ugliness: using a system property (a static non-final variable), because the better way to do it (a flag) would result in a much, much larger refactoring.
More ugliness: I have updated the minimal amount of tests. A lot of tests are still relying on the default value of MD5. Ideally, they need to be updated as well.
--
MOS_MIGRATED_REVID=139490836
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change updates WindowsFileSystem so it:
- retrieves the DosFileAttributes instead of the
BasicFileAttributes, because the latter does not
report junctions as directories
- uses just isJunction to decide if a file is a
symlink, doesn't look at whether it's a
directory (again because java.nio.File also
incorrectly reports junctions as
non-directories)
Fixes https://github.com/bazelbuild/bazel/issues/1850
--
MOS_MIGRATED_REVID=138187220
|