aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/syntax
Commit message (Collapse)AuthorAge
* [Skylark] Avoid unnecessary subList invocations.Gravatar Taras Tsugrii2018-08-14
| | | | | | | | | | It's better to not add `self` positional argument if it's not needed, instead of adding it and removing it laster on. `subList` invoaction results in more allocations and CPU overhead when using positional args. Closes #5812. PiperOrigin-RevId: 208687360
* Don't convert InterruptException to EvalException in struct field call ↵Gravatar Benjamin Peterson2018-08-08
| | | | | | | | | | | expressions. This is probably only a theoretical problem, since a blocking struct field is probably a very bad idea. Closes #5132. Change-Id: Ie84a78ab4d9ce215f2806ac49bf8911de6959930 PiperOrigin-RevId: 207902766
* Put main repo remapping behind a flag.Gravatar dannark2018-08-07
| | | | | RELNOTES: None PiperOrigin-RevId: 207801155
* C++: Change Skylark API whitelisting to be part of flag.Gravatar plf2018-08-03
| | | | | | | | This uses SkylarkSemantics now instead of the C++ configuration. The flag is: --experimental_cc_skylark_api_enabled_packages RELNOTES:none PiperOrigin-RevId: 207235431
* [Skylark] Optimize UserDefinedFunction.Gravatar Taras Tsugrii2018-08-02
| | | | | | | | | Java's iterator overhead is significant (2X slower than using indices), so it's worthwhile to use indices for hot code. Closes #5737. PiperOrigin-RevId: 207088309
* [Skylark] Speed up string.partition function.Gravatar Taras Tsugrii2018-08-02
| | | | | | | | | | | | | | | | According to JMH using `ImmutableList.of` or `Arrays.asList` is ~2X faster than using existing approach of creating an empty `ArrayList` with expected size and populating it using `add` method. This is most likely due to extra range and capacity checks. Returning `ImmutableList` instead of `ArrayList` avoids the need to copy it again in order to create a `SkylarkTuple`. These changes speed up the function by ~3X. Closes #5736. PiperOrigin-RevId: 207079605
* [Skylark] Improve Skylark interpreter performance.Gravatar Taras Tsugrii2018-08-02
| | | | | | | | | | | | | According to JMH and JIT assembly generated for iterators and index accesses, iterator methods like `hasNext` and `next` are not optimized away and result in 3-4X slower execution. Not using an iterator reduces Buck parse time for FB's internal Android apps by 7%+. Closes #5735. PiperOrigin-RevId: 207073078
* [Skylark] Avoid unnecessary allocations.Gravatar Taras Tsugrii2018-08-01
| | | | | | | | | Since this happens only every single Skylark method invocation CPU and memory usage quickly adds up and shows up on profiler samples a lot. Closes #5656. PiperOrigin-RevId: 206935520
* [Skylark] Move method invocation logic to MethodDescriptor.Gravatar Taras Tsugrii2018-07-31
| | | | | | | | | | | | | | | | | | This serves 2 purposes: - better encapsulation and domain model. - opens the door to optimizations like using `MethodHandle` instead of `Method` and caching. - performs one of the optimizations mentioned above - perform `setAccessible` only once instead of on every method invocation. JMH suggests that this saves ~5% of CPU time. Next steps are: - evaluate peformance improvements for some of the optimizations listed above - make PRs to apply optimizations that seem beneficial. Closes #5704. PiperOrigin-RevId: 206805670
* [Skylark] Cache isParamNamed computation.Gravatar Taras Tsugrii2018-07-31
| | | | | | | | | | When `isLegacyNamed` is `true`, `named` is considered to be `true` as well, so instead of going through an extra indirection and computation, just use `named` field to store combined result. Closes #5701. PiperOrigin-RevId: 206800781
* [Skylark] Avoid unnecessary copyOf invocations.Gravatar Taras Tsugrii2018-07-31
| | | | | | | | According to async-profiler, about 50% of `Tuple.getSlice` method invocation is spent in `ImmutableList.copyOf` which is completely unnecessary for cases when an `ImmutableList` instance is passed. Closes #5699. PiperOrigin-RevId: 206787490
* [Skylark] Size tuple slice sizes to avoid unnecessary allocations.Gravatar Taras Tsugrii2018-07-31
| | | | | | Closes #5698. PiperOrigin-RevId: 206780174
* [Skylark] Size argument list builder to avoid allocations.Gravatar Taras Tsugrii2018-07-30
| | | | | | | | | | | This change is focused on 2 things: - avoid creating builders in case they don't end up being used - create builders using the maximum expected size to avoid intermediate allocations to accommodate more elements Closes #5694. PiperOrigin-RevId: 206636046
* [Skylark] Use POJOs instead of dynamic proxies.Gravatar Taras Tsugrii2018-07-30
| | | | | | | | | | | | | | Java uses dynamically generated proxy classes to access annotation properties and their methods are ~7X slower than plain getters. According to async-profiler 50%+ of `convertArgumentList` method time is spent in dynamic proxy methods, so optimizing their performance makes sense. This also makes the model less anemic, since POJOs can actually provide business methods. Closes #5666. PiperOrigin-RevId: 206608812
* Cache default parameter values for skylark methods, and compute types of ↵Gravatar cparsons2018-07-25
| | | | | | | | | Param annotations only once. This change has been manually verified to greatly reduce analysis time (~50%) on very large builds, as it mitigates a previous regression brought on by the migration of @SkylarkSignature to @SkylarkCallable. RELNOTES: None. PiperOrigin-RevId: 206063684
* [Perf] Remove unnecessary ImmutableMap.copyOf invocation.Gravatar Taras Tsugrii2018-07-25
| | | | | | | | | `keyWordArgs` is already `ImmutableMap<String, Object>` and `copyOf` contains enough overhead to show up on parse profiles. Closes #5651. PiperOrigin-RevId: 206051798
* Update commentsGravatar laurentlb2018-07-24
| | | | | | | Comments are misleading, as discussed on https://github.com/bazelbuild/bazel/pull/5305#issuecomment-396288826 RELNOTES: None. PiperOrigin-RevId: 205841782
* If a dictionary is used as a general set, the keys should be mapped to ↵Gravatar Googler2018-07-18
| | | | | | | | | | `True` instead of `None`. dict has a get() method that defaults to `None`. Checking for a key in the dictionary with get() will always return `None` in the given example. Using `True` is better. RELNOTES: None. PiperOrigin-RevId: 205052981
* [Skylark] Make range function lazy.Gravatar Taras Tsugrii2018-07-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | range used to use MutableList which would eagerly allocate an array list with all range elements, which is not efficient for very large ranges or when only a small number of its elements are used. This implementation uses a constant amount of RAM and computes a value for each requested index. For the following Skylark snippet: ``` def check_content(t): if t == []: return t return False def modulo(n): return n % 797 N = 10000000 [check_content(i) for i in range(N)] [check_content(i) for i in range(N)] [modulo(i) for i in range(N)] [modulo(i) for i in range(N)] ``` the total runtime goes from ``` $ time bazel-bin/src/main/java/com/google/devtools/skylark/Skylark test.bzl bazel-bin/src/main/java/com/google/devtools/skylark/Skylark test.bzl 93.09s user 1.67s system 316% cpu 29.930 total ``` to ``` $ time bazel-bin/src/main/java/com/google/devtools/skylark/Skylark test.bzl bazel-bin/src/main/java/com/google/devtools/skylark/Skylark test.bzl 31.45s user 0.86s system 179% cpu 17.974 total ``` which reflects the reduced system time (fewer allocations) and performance. Closes #5240. PiperOrigin-RevId: 204918577
* Add --incompatible_disable_deprecated_attr_params to disable some deprecated ↵Gravatar cparsons2018-07-16
| | | | | | | parameters of the skylark attr module. RELNOTES: None. PiperOrigin-RevId: 204797954
* Remove SkylarkCallable.mandatoryPositionalsGravatar cparsons2018-07-13
| | | | | RELNOTES: None. PiperOrigin-RevId: 204505003
* Improve logging for parsing skylark files.Gravatar twerth2018-07-09
| | | | | | | Also add preconditions check. RELNOTES: None. PiperOrigin-RevId: 203731929
* Add an interner to LineNumberTable for during deserialization.Gravatar mjhalupka2018-06-29
| | | | PiperOrigin-RevId: 202690640
* [java_common.compile] Always generate a source jar.Gravatar elenairina2018-06-29
| | | | | | | | | | | | | | | [Rolling forward https://github.com/bazelbuild/bazel/commit/c4e128e2c6d8cacaeba034d6a3195796d50f1745] java_common.compile doesn't generate the output source jar when a source jar is the only input for the compilation. This is wrong because the source jar can include APT generated sources. It is also inconsistent with java_library and leads to inconsistent Skylark rules where a declared output will not always have a generating action. This new behavior is guarded by a new flag --incompatible_generate_javacommon_source_jar. RELNOTES: None. PiperOrigin-RevId: 202648346
* [Skylark] Allow tuples as first argument of str.{starts,ends}withGravatar Yannic Bonenberger2018-06-28
| | | | | | | | Closes #5307 Closes #5455. PiperOrigin-RevId: 202567483
* Add --incompatible_disable_data_transition for 'cfg = "data"' Skylark callsGravatar gregce2018-06-27
| | | | PiperOrigin-RevId: 202400312
* Remove obsolete function.Gravatar laurentlb2018-06-27
| | | | | RELNOTES: None. PiperOrigin-RevId: 202325860
* Handle evaluation of statements, not just expressions.Gravatar brendandouglas2018-06-26
| | | | | | | | | | | | | | | Also handle statements in conditional breakpoints. This is more consistent with other common debuggers (e.g. java, python). Calls Parser#parseStatement with local parsing level, so some statement types aren't handled (e.g. load statements), which is broadly consistent with other debuggers. Assignment, augmented assignment, and return statements return a non-None value, and simple expression statements still return the result of evaluating the expression. TAG_CHANGE_OK=This proto has never yet been used TYPE_CHANGE_OK=This proto has never yet been used PiperOrigin-RevId: 202135678
* Make @SkylarkCallable.name mandatory.Gravatar cparsons2018-06-22
| | | | | RELNOTES: None. PiperOrigin-RevId: 201748802
* Audit usages of Label.getRelative and update to Label.getRelativeWithRemappingGravatar dannark2018-06-21
| | | | | RELNOTES: None PiperOrigin-RevId: 201617188
* Create a method for initializing build API libraries using object instances ↵Gravatar cparsons2018-06-21
| | | | | | | | | instead of object classes. This allows the caller to, for example, initialize the global library before adding it to a global frame. The skylark interpreter will no longer require that global libraries have a zero-arg constructor. RELNOTES: None. PiperOrigin-RevId: 201563608
* Skylark debugging protocol: only track paused or stepping threads.Gravatar brendandouglas2018-06-21
| | | | | | | | | | | | | | | | | | | | | | | | - remove blaze-side thread creation/destruction hooks - remove ListThreads, ThreadStarted, ThreadEnded events from protocol - don't track unpaused, not stepping threads in ThreadHandler The threading model didn't provide useful information -- in practice, users never want to list currently-running threads, and the debug client doesn't need to list currently-paused threads, since it receives each ThreadPaused and ThreadContinued event, so its model is always up to date. Not tracking thread creation/destruction greatly simplifies the API, and reduces the blaze-side hooks -- it was also only semi-implemented, as plenty (/most?) skylark threads were never registered. The biggest cost to removing this is lack of nice thread names. In practice, I think the benefits greatly outweigh this cost. If it ever becomes a problem, there are other lighter-weight ways of communicating thread names. TAG_CHANGE_OK=This proto has never yet been used TYPE_CHANGE_OK=This proto has never yet been used PiperOrigin-RevId: 201532462
* Allow structField callables to specify useSkylarkSemantics, useLocation, and ↵Gravatar cparsons2018-06-20
| | | | | | | | | | useEnvironment Unfortunately this doesn't work for all callers, namely NativeInfo objects, as they may have structField callables invoked from contexts that have no environment available. RELNOTES[INC]: Skylark structs (using struct()) may no longer have to_json and to_proto overridden. PiperOrigin-RevId: 201376969
* Make things easier to serialize: make constants constant, tag some lambdas. ↵Gravatar janakr2018-06-19
| | | | | | Get rid of a useless tag, because the Function being tagged is a concrete class, so won't be serializable. Will deal with it in a follow-up. Implement equality for BazelInfo. PiperOrigin-RevId: 201199255
* Remove LIPO supportGravatar hlopko2018-06-15
| | | | | RELNOTES: Support for LIPO has been fully removed. PiperOrigin-RevId: 200724578
* Move remaining BazelLibrary skylark functions to MethodLibraryGravatar cparsons2018-06-15
| | | | | | | | | Ultimately, we'll need to make the call on whether these functions belong as part of the build API or as part of skylark builtins. For now, we keep them as skylark builtins. (In either case, we'll want to migrate to @SkylarkCallable, but that's for a later change) RELNOTES: None. PiperOrigin-RevId: 200723605
* Skylark Pretty Printer: shorten empty listGravatar Klaus Aehlig2018-06-14
| | | | | | | | | | | | | | When pretty printing a Skylark value, lists are presented as the opening bracket on a line by itself, each entry on its own line, and the closing bracket again on its own line. While this generally improves readability, for the empty list this is not the case, as the expression [] can easily be understood at a glance. In fact, the additional line even makes the outer structure harder to see, as it is spread over even more lines. Therefore, shorten the printing of the empty list to be on a single line. Change-Id: I032d1550b1f99bce47dbec7e77a4d5c6656d78a1 PiperOrigin-RevId: 200558784
* Skylark debugger: Fix NPE listing frames.Gravatar brendandouglas2018-06-12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Environment$Continuation#caller is most definitely nullable in practice. I suspect it's a bug in skylark itself, but for now, just properly mark it nullable and handle it in the debugger. Stack trace from NPE: Caused by: java.lang.NullPointerException at com.google.devtools.build.lib.syntax.Environment.listFrames(Environment.java:1197) at com.google.devtools.build.lib.syntax.Environment.listFrames(Environment.java:81) at com.google.devtools.build.lib.skylarkdebug.server.ThreadHandler.listFrames(ThreadHandler.java:236) at com.google.devtools.build.lib.skylarkdebug.server.ThreadHandler.getThreadProto(ThreadHandler.java:345) at com.google.devtools.build.lib.skylarkdebug.server.ThreadHandler.pauseCurrentThread(ThreadHandler.java:289) at com.google.devtools.build.lib.skylarkdebug.server.ThreadHandler.pauseIfNecessary(ThreadHandler.java:203) at com.google.devtools.build.lib.skylarkdebug.server.SkylarkDebugServer.pauseIfNecessary(SkylarkDebugServer.java:158) at com.google.devtools.build.lib.skylarkdebug.server.SkylarkDebugServer$DebugAwareEval.exec(SkylarkDebugServer.java:262) at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91) at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:462) at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:440) at com.google.devtools.build.lib.analysis.skylark.SkylarkRuleConfiguredTargetUtil.lambda$buildRule$1(SkylarkRuleConfiguredTargetUtil.java:105) at com.google.devtools.build.lib.skylarkdebug.server.SkylarkDebugServer.runWithDebugging(SkylarkDebugServer.java:142) at com.google.devtools.build.lib.syntax.DebugServerUtils.runWithDebuggingIfEnabled(DebugServerUtils.java:70) at com.google.devtools.build.lib.analysis.skylark.SkylarkRuleConfiguredTargetUtil.buildRule(SkylarkRuleConfiguredTargetUtil.java:100) PiperOrigin-RevId: 200229036
* Remap repository names inside load statements in BUILD files if the ↵Gravatar dannark2018-06-12
| | | | | | | | | | | | | | | | | | | repository name is remapped. For example if main/WORKSPACE contains: local_repository( name = "a", path = "../a", repo_mapping = {"@x" : "@y"}, ) a/BUILD load("@x//:sample.bzl", "sample") Then the load in a/BUILD will be resolved as "@y//:sample.bzl" RELNOTES: None PiperOrigin-RevId: 200227431
* ReformatingGravatar laurentlb2018-06-11
| | | | | | | | Switch statements were poorly formatted. Fixing it in a separate commit so that it doesn't clutter the diff. RELNOTES: None. PiperOrigin-RevId: 200062930
* Add AutoProfiler-like API to ProfilerGravatar ulfjack2018-06-11
| | | | | | - migrate all startTask/completeTask pairs to the new API PiperOrigin-RevId: 200038703
* Use Identifiers instead of StringsGravatar Taras Tsugrii2018-06-08
| | | | | | | | | | | | The high level summary of the changes: - use `Identifier` instead of `name` in `Keyword` and `Parameter`. - construct `Identifier` through a factory method in case future interning is desired. These changes are in preparation for using `Identifier` instead of `name` for environment lookups. Closes #5304. PiperOrigin-RevId: 199869171
* Add the skylark debugging options to the recognized 'build' options.Gravatar brendandouglas2018-06-08
| | | | | | | | | | Ensure debugging is turned off (and the server socket closed) when the command finishes, even if the blaze server is shut down. Finally, unpause all threads as part of shutting down the debug server, and also shut down the server if the client connection is lost. PiperOrigin-RevId: 199863623
* Update documentation about dict iteration orderGravatar laurentlb2018-06-08
| | | | | | | | | | | The information was redundant. https://github.com/bazelbuild/bazel/commit/d605a62eb9908fe44c89a2ed10f88f633c4dcb3e modified only one place, which created a contradiction. We can just remove the outdated one. #5352 RELNOTES: None. PiperOrigin-RevId: 199780557
* Add an initial skylark debug server implementation.Gravatar Googler2018-06-07
| | | | | | | | | | | | Not intending this to be a complete implementation yet. Among the things still to do: - officially add support for debugging aspects, rules, etc., with corresponding tests. - handle breakpoints at return statements (an edge case which bypasses Eval). PiperOrigin-RevId: 199692670
* Fix a few generated docs.Gravatar Sam Rawlins2018-06-07
| | | | | | | | | | | | | | In particular, * fix sentences with no space after the period, like [here]([] * switch backticks to `<code>`, like [here]([] * link `ctx.var` [here]([] * Add periods to the end of paragraphs, like [here]([] * Format a list [here]([] Closes #5337. PiperOrigin-RevId: 199631585
* Add flag --experimental_enable_repo_mapping to gate usage of repo_mapping ↵Gravatar dannark2018-06-05
| | | | | | | functionality in WORKSPACE rules. RELNOTES: None PiperOrigin-RevId: 199332092
* Stop allocating new tokens in the lexerGravatar laurentlb2018-06-05
| | | | | | | | | | There's only one Token and it gets reused. This reduces the memory usage of the lexer. Parsing time seems to be 5%-10% faster with this change on a large file. This makes little difference on the overall performance of Bazel though. RELNOTES: None. PiperOrigin-RevId: 199310860
* Get rid of the tokens queue in the lexerGravatar laurentlb2018-06-04
| | | | | | | Next step will be to skip token allocation. RELNOTES: None. PiperOrigin-RevId: 199121625
* Make tools in action inputs an error.Gravatar tomlu2018-06-02
| | | | | | | | | | Supporting tools in inputs introduces a slow linear scan. Such tools should be moved to the 'tools' attribute. If --incompatible_no_support_tools_in_action_inputs is set the inputs are scanned, but a helpful error message is issued to the user. Eventually we will remove the slow scanning. Errors will surface in the execution phase instead of during analysis, and the resulting error messages will be less obvious. RELNOTES: None RELNOTES[INC]: With --incompatible_no_support_tools_in_action_inputs enabled, Skylark action inputs are no longer scanned for tools. Move any such inputs to the newly introduced 'tools' attribute. PiperOrigin-RevId: 198996093
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-01 14:34:16 +0000