aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools
Commit message (Collapse)AuthorAge
* Open-source ImmutableMapCodec and make it able to handle arbitrary ↵Gravatar janakr2018-03-10
| | | | | | | | | | keys/values without injecting the codecs. Also allow it to handle ImmutableSortedMap, since we were always silently degrading to ImmutableMap for objects that weren't declared as ImmutableSortedMap, and there's no good way to handle non-natural comparators. This will lead to runtime failures if a class actually needs an ImmutableSortedMap with a different comparator than the natural one. This changes the semantics of ImmutableMap serialization. Previously, we went off the declared type. In the case of a declared ImmutableSortedMap, we ignored the comparator, potentially leading to incorrect serialization (new test added in AutoCodecProcessorTest that would have caught that). Moreover, declared ImmutableMaps were deserialized as ImmutableMaps even if they were actually ImmutableSortedMaps. Now, we preserve the ordering unconditionally, and preserve the type if possible. I think this is a better state to be in. This is needed to kill off MemoizingCodec, since MemoizingCodec has an ImmutableMapMemoizingCodec, which I want to get rid of in favor of this. PiperOrigin-RevId: 188619637
* Replace ConfiguredTargetAndData#getConfiguredTarget()#getConfiguration() ↵Gravatar janakr2018-03-10
| | | | | | with ConfiguredTargetAndData#getConfiguration(). Done using intellij structural replace. PiperOrigin-RevId: 188618282
* Make memoizing serialize and deserialize methods that take a codec ↵Gravatar janakr2018-03-10
| | | | | | package-private. PiperOrigin-RevId: 188610579
* Explicitly don't serialize/deserialize Package objects in ↵Gravatar janakr2018-03-10
| | | | | | ConfiguredTargetValue. The nested set of packages isn't usually set when serializing/deserializing, but it's good to be explicit, especially in tests. PiperOrigin-RevId: 188605043
* Update links to examplesGravatar brandjon2018-03-10
| | | | | | | See changes in https://github.com/bazelbuild/examples/pull/56 RELNOTES: None PiperOrigin-RevId: 188601311
* Add BuildConfiguration to ConfiguredTargetAndTarget and rename it to ↵Gravatar janakr2018-03-10
| | | | | | ConfiguredTargetAndData. We want to get BuildConfiguration out of ConfiguredTarget because it uses >800K when serialized. PiperOrigin-RevId: 188600002
* ActionOwner references only BuildEvent instead of full BuildConfigurationGravatar shahan2018-03-10
| | | | | | This reduces the size of its serialized representation. PiperOrigin-RevId: 188597127
* remote/http: support refresh of oauth2 tokens in the remote cache.Gravatar Jakob Buchgraber2018-03-10
| | | | | | Closes #4622. PiperOrigin-RevId: 188595430
* Allow ObjectCodecRegistry to handle MemoizingCodecs. Initially this is just ↵Gravatar janakr2018-03-09
| | | | | | | | | | | | | | | | Skylark memoization, but we may extend in future to handle more than just Skylark this way. In fact, we probably want most of our ObjectCodecs to be MemoizingCodecs that can efficiently fall back to ObjectCodec if not using memoization (maybe coming in a follow-up). At a high level, this CL merges the functionality of MemoizingCodecMap into ObjectCodecRegistry, adds auto-registration of MemoizingCodec, and uses that to get rid of a lot of codecs that were just delegating. The big one to get rid of there is SkylarkValueCodec: all of its delegation duties are implicitly now in ObjectCodecRegistry and friends. One danger with this CL is that many of the features of Skylark serialization are only being tested in unit tests, which had to be reworked as part of this change. I don't think we've lost any coverage, but I could be wrong. SkylarkValueCodec had a bunch of methods that were effectively test-only, which made it easier to remove. The plan is to provide a Memoizer.Serializer inside the SerializationContext. At the top level, it will be a DUMMY_SERIALIZER that does no memoization, but a MemoizingCodec can do context = context.ensureMemoizing() which will recreate the context with a true memoizing serializer. Then all references to the Serializer in codec code can be cleaned up, and the MemoizingCodec and ObjectCodec signatures will be the same. At that point, we can make all ObjectCodecs compatible with memoization by default (with strategy MEMOIZE_AFTER), and add a "memoize" boolean to @AutoCodec. That should allow us to have full interoperability between all codecs. This CL also makes CodecScanner deterministic in the order of classes that it processes (there was a lurking bug here where constants must be deterministically ordered but that wasn't enforced at all). PiperOrigin-RevId: 188559983
* Remove cycle error reporting from query results since they're not strictly ↵Gravatar juliexxia2018-03-09
| | | | | | relevant and only trigger when the implicit or explicit max depth > 20 which is confusing. PiperOrigin-RevId: 188559702
* Use RuleContext.getConfiguration() instead of ↵Gravatar janakr2018-03-09
| | | | | | | | RuleConfiguredTarget.getConfiguration(), since we have it handy. We'd like to get rid of BuildConfiguration from RuleConfiguredTarget. PiperOrigin-RevId: 188545048
* Support for passing options to LTO Backend actions onlyGravatar Googler2018-03-09
| | | | | | | | This is very useful for debugging and performance tuning. RELNOTES[NEW]: Adds --ltobackendopt and --per_file_ltobackendopt for passing options to ThinLTO LTO backend compile actions only. PiperOrigin-RevId: 188521075
* Add the ability to reconstruct a BuildOptions from the diff with another ↵Gravatar janakr2018-03-09
| | | | | | BuildOptions. Motivation is that the diffs are likely to be much smaller than the actual BuildOptions objects themselves, so in places we need a BuildOptions (I'm looking at you, BuildConfigurationValue.Key), we can instead store a diff, reconstructing the BuildOptions object itself on demand when needed. PiperOrigin-RevId: 188511251
* Improve the warning for malformed recursive configs.Gravatar ccalvarin2018-03-09
| | | | | | | | | Unlike in fixed-point (legacy) expansion of configs, with --expand_configs_in_place we do not use the options parser to parse each config-definition default override - we first find the full expansion and then expand it in place of the original --config=value instance. For this reason though, we don't support space-separation of recursive configs and their values. The old warning for this was confusing though, and did not provide much guidance. This should be better, now the warning specifies which config is malformed, in what file, and that it expects the "=" character. RELNOTES: None. PiperOrigin-RevId: 188509060
* Enable static_linking_mode or dynamic_linking_mode features from cc_binaryGravatar hlopko2018-03-09
| | | | | RELNOTES: CppRules: cc_binary/cc_test now enable 'static_linking_mode' or 'dynamic_linking_mode'. PiperOrigin-RevId: 188482267
* tests: fix flaky RecursiveFSTraversalFunctionTestGravatar Laszlo Csomor2018-03-09
| | | | | | | | | | | | | | | | | | Fix the testTraversalOfGeneratedDirectory method in RecursiveFilesystemTraversalFunctionTest that was flaky on Windows. The fix is to wait so that changes to files in the InMemoryFileSystem will have observable effects on the file ctimes. Depends on https://github.com/bazelbuild/bazel/pull/4787 Fixes https://github.com/bazelbuild/bazel/issues/4755 Closes #4788. PiperOrigin-RevId: 188470080
* tests: fix flaky SkyframeAwareActionTestGravatar Laszlo Csomor2018-03-09
| | | | | | | | | | | | | Fix testCacheBypassingActionWithMtimeChangingInput in SkyframeAwareActionTest by ensuring that enough time elapses between file updates so their effects are observable on the file's ctime. Fixes https://github.com/bazelbuild/bazel/issues/4755 Closes #4787. PiperOrigin-RevId: 188458542
* Optimize GC churn of parameter bindings performed before each user defined ↵Gravatar nharmata2018-03-08
| | | | | | | | | | | | | | | function call. This is the 2nd attempt at this commit. The first attempt (https://github.com/bazelbuild/bazel/commit/f1013485d41efd8503f9d4f937e17d1b4bc91ed3) was rolled back because it introduced the following two bugs: (1) The side effects of Environment#enterScope are relevant: it creates and stores a new Continuation that has a reference to the set currently referenced by 'knownGlobalVariables', and then overwrites the value of the variable. When there are e.g. nested function calls, 'knownGlobalVariables' will be wrong in the Environment used to stage the inner call (see the added test for an example). (2) The finally block in UserDefinedFunction#call assumes the env.enterScope was called. Because of the EvalException (incorrectly) thrown due to (1), this is no longer true. I restructured the code such that (2) isn't possible and I also added a unit test that would have caught the two bugs. In my first attempt, I was doing too much - I was also trying to save the CPU-costs in the env.update call (dispatches to the just-created lexical frame, and calls LexicalFrame#put, which does an unnecessary mutability sanity check, etc) and in doing so completely missed the above bugs. Sorry. RELNOTES: None PiperOrigin-RevId: 188411737
* Add the flag --experimental_import_deps_checking to control the dependencyGravatar cnsun2018-03-08
| | | | | | | checking for aar_import and java_import targets. RELNOTES:None. PiperOrigin-RevId: 188399775
* Remove all dynamic type checking overhead for struct field accesses. These ↵Gravatar nharmata2018-03-08
| | | | | | | checks are unnecessary by construction; see the codepaths that construct SkylarkInfo instances. RELNOTES: None PiperOrigin-RevId: 188373688
* Tag SkylarkComputedDefault with @AutoCodecGravatar mjhalupka2018-03-08
| | | | PiperOrigin-RevId: 188367892
* Tag SkylarkType with @AutoCodec.Gravatar mjhalupka2018-03-08
| | | | PiperOrigin-RevId: 188367672
* As a micro-optimization, inline the "Callstack.enabled" guard. The java-land ↵Gravatar nharmata2018-03-08
| | | | | | | | | function call overhead of the morally no-op Callstack#push/pop was profiled to be ~1.4% CPU in a benchmark of loading a BUILD file that was particularly heavy in Skylark function calls. Alternatives considered: writing code that I hoped would be more amenable to the JIT choosing to inline the function call. I couldn't get this to work. RELNOTES: None PiperOrigin-RevId: 188350132
* Intern CollectTargetsInPackagKey. Bug from ↵Gravatar janakr2018-03-08
| | | | | | https://github.com/bazelbuild/bazel/commit/5fb2a487e53cc3d80e3654d5b63d062f7f70588b. PiperOrigin-RevId: 188348546
* Add an interface to interact with sandboxfs.Gravatar jmmv2018-03-08
| | | | | | | | | | | | | | | | The new SandboxfsProcess interface allows interacting with sandboxfs. There are two implementations: RealSandboxfsProcess, which spawns the sandboxfs binary, and FakeSandboxfsProcess, which mimics what sandboxfs does but using symlinks and is intended for testing purposes only. The RealSandboxfsProcess implementation works but still carries many TODOs. The most "painful" one may be that the test requires manual invocation because we do not yet have an easy way to integrate with sandboxfs. That will be solved later on; for now this is sufficient for initial testing. RELNOTES: None. PiperOrigin-RevId: 188347393
* Remove unnecessary I/O operations from handling remotely executed actions ↵Gravatar olaola2018-03-08
| | | | | | | | (fixes performance regression #4749). Also adding Skylark tests for input/output directories. TESTED=locally RELNOTES: Fix performance regression PiperOrigin-RevId: 188346410
* Remove unnecessary space in error message.Gravatar ajmichael2018-03-08
| | | | | RELNOTES: None PiperOrigin-RevId: 188341095
* update to MethodDescriptor methods instead of fieldsGravatar Carl Mastrangelo2018-03-08
| | | | | | | | @buchgr Closes #4790. PiperOrigin-RevId: 188332795
* C++: Marks Skylark exposed as not documented.Gravatar plf2018-03-08
| | | | | | | This is to avoid listing them in the Bazel docs for now. RELNOTES:none PiperOrigin-RevId: 188295824
* Fix PackageFunction's call to Package.Builder.Helper#onLoadingComplete to passGravatar nharmata2018-03-07
| | | | | | | | | | along the wall time of the load, even when the package in question was in PackageFunction's internal cache (e.g. the current #compute call is a PackageFunction Skyframe restart). Also clarify the intent of the 'loadTimeMs' param in #onLoadingComplete. RELNOTES: None PiperOrigin-RevId: 188253198
* Automated rollback of commit f1013485d41efd8503f9d4f937e17d1b4bc91ed3.Gravatar nharmata2018-03-07
| | | | | | | | | | | | | *** Reason for rollback *** Introduced a bug in skylark that caused intellij TAP test to fail. The bug was not caught by any skylark/blaze/bazel tests. *** Original change description *** Optimize GC churn of parameter bindings performed before each user defined function call. RELNOTES: None PiperOrigin-RevId: 188249713
* Fixed missing C standard library functions with NDK r15 and r16Gravatar jingwen2018-03-07
| | | | | | | | | Certain C++ stdlib wrapper headers are unable to find their C counterparts (e.g. math.h for cmath) that are in %ndk%/sysroot/usr/include. This is because the -isystem for the include path was added with addCompilerFlag instead of addUnfilteredCxxFlag. The former is filtered out when compiling C code, whereas the latter keeps it. Fixes https://github.com/bazelbuild/bazel/issues/4777 RELNOTES: None. PiperOrigin-RevId: 188241254
* Fix stale comment about selector entry orderGravatar mschaller2018-03-07
| | | | | RELNOTES: None. PiperOrigin-RevId: 188221536
* Create a new version of BuildTool that holds all cquery logic.Gravatar juliexxia2018-03-07
| | | | PiperOrigin-RevId: 188212286
* Allow passing location, ast, and environment to @SkylarkCallable methodsGravatar cparsons2018-03-07
| | | | | RELNOTES: None. PiperOrigin-RevId: 188201686
* Optimize GC churn of parameter bindings performed before each user defined ↵Gravatar nharmata2018-03-07
| | | | | | | function call. RELNOTES: None PiperOrigin-RevId: 188199514
* Added named=true to output_group.group_nameGravatar cparsons2018-03-07
| | | | | | | This was migrated incorrectly, as there's a semantic difference between @Param for @SkylarkSignature and for @SkylarkCallable, it would appear. RELNOTES: None. PiperOrigin-RevId: 188190409
* Remove internal-only startup option from documentation.Gravatar ajmichael2018-03-07
| | | | | RELNOTES: None PiperOrigin-RevId: 188179200
* Refactoring FileTransport to use the AsynchronousFileOutputStream.Gravatar olaola2018-03-07
| | | | | | | | Very thin wrapper, nothing except swallow+log all errors. TESTED=presubmit RELNOTES: None PiperOrigin-RevId: 188177872
* Add a binary to CppCompileAction that is responsible for grepping header ↵Gravatar ajmichael2018-03-07
| | | | | | | files for include statments. This binary is currently only used for an internal feature - but that feature may be supported externally eventually. RELNOTES: None PiperOrigin-RevId: 188173513
* Add option to dump command lines of actions in dump.Gravatar twerth2018-03-07
| | | | | RELNOTES: None PiperOrigin-RevId: 188164754
* Don't use computeIfAbsent in action graph dumping.Gravatar twerth2018-03-07
| | | | | | | | | | | | | | | computeIfAbsent may throw (on best effort basis) a ConcurrentModificationException if you change the underlying Map in the computeIfAbsent call, for example when you call computeIfAbsent recursively in the mappingFunction. NestedSets are an inherent recursive structure, so we need to call computeIfAbsent recursively in the action graph dump. Since computeIfAbsent doesn't throw the exception all the time, it's hard to come up with a reliable test case. RELNOTES: None PiperOrigin-RevId: 188151283
* C++: Makes sure ltoBitcodeFiles get merged when merging compilation outputs.Gravatar plf2018-03-07
| | | | | RELNOTES: PiperOrigin-RevId: 188149648
* C++ providers: Removes C++ providers from JavaWrapCcGravatar plf2018-03-07
| | | | | | | They are now wrapped by a JavaWrapCc specific provider. RELNOTES:none PiperOrigin-RevId: 188140909
* Post PatternExpandingError from the skyframe target pattern evaluator as needed.Gravatar Benjamin Peterson2018-03-07
| | | | | | | | | This fixes https://github.com/bazelbuild/bazel/issues/4731. Also, add test coverage for posting of ParsingFailedEvent and handling of cycles in LoadingPhaseRunnerTest. Change-Id: I88c9d33417b9c3c7a06c92a6137d58f37b991b0c PiperOrigin-RevId: 188138972
* Use -isystem instead of -I for NDK header search paths.Gravatar jingwen2018-03-06
| | | | | | | | | | | | | | This suppresses warnings from the NDK headers from users. This also fixes the correct include search paths, because -isystem dirs are searched _after_ -I dirs. https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html#Directory-Options "You can use -I to override a system header file, substituting your own version, since these directories are searched before the standard system header file directories. However, you should not use this option to add directories that contain vendor-supplied system header files; use -isystem for that. The -isystem and -idirafter options also mark the directory as a system directory, so that it gets the same special treatment that is applied to the standard system directories." RELNOTES[FIX]: Fixed NDK r15/r16 system include search paths to use -isystem instead of -I. PiperOrigin-RevId: 188111072
* Expose an actions provider on RuleConfiguredTarget instances.Gravatar cparsons2018-03-06
| | | | | | | | | Given a target (for example from a skylark aspect), one will be able to access a list of actions that the target generated using "target.actions". This is without additional memory footprint. Actions themselves are not fully exposed to skylark (and thus there isn't much meaning to gather from them in skylark yet). Access methods will follow soon. RELNOTES: None. PiperOrigin-RevId: 188098079
* Adding async proto or text logging utility class.Gravatar olaola2018-03-06
| | | | | | | WANT_LGTM=buchgr TESTED=unit tests, 500 runs per test RELNOTES: None PiperOrigin-RevId: 188093043
* @AutoCodec File{State}Value.Gravatar janakr2018-03-06
| | | | PiperOrigin-RevId: 188085895
* Make the builtins registry thread-safeGravatar brandjon2018-03-06
| | | | | | | | | | | | | | It was previously assumed that safety wasn't needed because 1) all builtins should be registered in static initializer blocks, and 2) all retrievals should occur during Skylark evaluation, after static initialization completes. It turns out these assumptions aren't actually true (Who would've thunk it!). SkylarkActionFactory has been observed to be initialized as late as analysis time, and retrievals occur as early as constructing a PackageFactory (when scanning the native module). The failure mode is particularly ugly: Random Skylark method lookups will fail non-deterministically. This change guards against this by making the builtins registry implement a form of freezing. Before freezing, reads and writes are allowed and are synchronized. After freezing, only reads are allowed and they are unsynchronized for performance. BlazeRuntime is responsible for flipping the bit, and for ensuring classes like SkylarkActionFactory run their initialization by that point. Unit tests don't need to worry, since they just stay unfrozen and synchronized throughout. RELNOTES: None PiperOrigin-RevId: 188080136
generated by cgit v1.2.3 (git 2.39.1) at 2024-10-20 00:29:25 +0000