aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/common
Commit message (Collapse)AuthorAge
* Add a <code>getSkylarkOptions</code> helper function to all OptionsProviders.Gravatar juliexxia2018-08-13
| | | | | | This is to allow OptionsProviders to access the new skylark options which aren't held in a fragment class like native options and thus can't be accessed with the traditional <code>get</code> method. Currently this function isn't called anywhere and would just return an empty map. But in the future it will return a map of skylark flag names to values. PiperOrigin-RevId: 208552510
* Modify --subcommands to allow pretty printing the arguments of a subcommandGravatar ahumesky2018-07-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | as a list, rather than as a single line (i.e., newline delimited rather than space delimited). Before: SUBCOMMAND: # //src/main/java/com/google/devtools/build/lib:string_util [action 'Building src/main/java/com/google/devtools/build/lib/libstring_util.jar (2 source files) [for host]'] (cd /tmp/devbazel_output_base/execroot/io_bazel && \ exec env - \ LC_CTYPE=en_US.UTF-8 \ external/embedded_jdk/bin/java -XX:+UseParallelOldGC -XX:-CompactStrings '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED' '--add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' '--patch-module=java.compiler=external/bazel_tools/third_party/java/jdk/langtools/java_compiler.jar' '--patch-module=jdk.compiler=external/bazel_tools/third_party/java/jdk/langtools/jdk_compiler.jar' '--add-opens=java.base/java.nio=ALL-UNNAMED' -jar external/bazel_tools/tools/jdk/JavaBuilder_deploy.jar @bazel-out/host/bin/src/main/java/com/google/devtools/build/lib/libstring_util.jar-2.params) After: SUBCOMMAND: # //src/main/java/com/google/devtools/build/lib:string_util [action 'Building src/main/java/com/google/devtools/build/lib/libstring_util.jar (2 source files) [for host]'] (cd /tmp/devbazel_output_base/execroot/io_bazel && \ exec env - \ LC_CTYPE=en_US.UTF-8 \ external/embedded_jdk/bin/java \ -XX:+UseParallelOldGC \ -XX:-CompactStrings \ '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' \ '--add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED' \ '--add-exports=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED' \ '--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' \ '--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED' \ '--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED' \ '--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED' \ '--add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' \ '--patch-module=java.compiler=external/bazel_tools/third_party/java/jdk/langtools/java_compiler.jar' \ '--patch-module=jdk.compiler=external/bazel_tools/third_party/java/jdk/langtools/jdk_compiler.jar' \ '--add-opens=java.base/java.nio=ALL-UNNAMED' \ -jar \ external/bazel_tools/tools/jdk/JavaBuilder_deploy.jar \ @bazel-out/host/bin/src/main/java/com/google/devtools/build/lib/libstring_util.jar-2.params) RELNOTES: --subcommands can now take a "pretty_print" value ("--subcommands=pretty_print") to print the arguments of subcommands as a list for easier reading. PiperOrigin-RevId: 206213009
* Log warning cleanupGravatar neerajen2018-05-31
| | | | | | | Setting --config in .blazerc that overrode previous options in .blazerc was triggering unnecessary warnings. Fixed by only logging warnings when a user attempts to override an option specified on the command line. RELNOTES: None. PiperOrigin-RevId: 198756160
* Allow --worker_max_instances to take MnemonicName=value to specify max for ↵Gravatar Googler2018-05-03
| | | | | | | each named worker. RELNOTES: Allow --worker_max_instances to take MnemonicName=value to specify max for each worker. PiperOrigin-RevId: 195244295
* Clean up code that directly imports nested classes like Builder, Entry, etc.Gravatar jcater2018-05-02
| | | | PiperOrigin-RevId: 195100670
* Clean up code that directly imports nested classes like Builder, Entry, etc.Gravatar jcater2018-05-01
| | | | PiperOrigin-RevId: 195040539
* Remove use of bare Immutable{List,Map,Set} Builder classes.Gravatar jcater2018-04-20
| | | | | | Always use the more-qualified class name for clarity at the site of use. There are too many classes named Builder. PiperOrigin-RevId: 193649193
* Make attempting to change --config in invocation policy an error.Gravatar ccalvarin2018-04-17
| | | | | | | It will not work as expected, since config is already expanded by this point in options processing. RELNOTES: None. PiperOrigin-RevId: 193196664
* Remove alphabetical sorting of options in the canonical list.Gravatar ccalvarin2018-04-09
| | | | | | | This was broken for --config. Doing this properly requires keeping the order in which the options were given, which could be done either by filtering the ordered list according to which values affect the final outcome or by tracking the order correctly. I picked the later: the option order was not explicitly tracked for expansions before but now it is. RELNOTES: canonicalize-flags no longer reorders the flags PiperOrigin-RevId: 192132260
* Remove category checking from incompatible changes.Gravatar ccalvarin2018-03-30
| | | | | | | String categories are deprecated, replace this special-cased value with a specific OptionMetadata tag, TRIGGERED_BY_ALL_INCOMPATIBLE_CHANGES. RELNOTES: None. PiperOrigin-RevId: 191069412
* Support source versions newer than 8 in Bazel's annotation processorsGravatar cushon2018-03-29
| | | | | | This quiets some build warnings. PiperOrigin-RevId: 190958692
* Fix invocation policy's handling of the null default when filtering values.Gravatar ccalvarin2018-03-01
| | | | | | | For a filter on option values (either by whitelist, allow_values, or blacklist, disallow_values), one of the options for what to do when encountering a disallowed value is to replace it with the default. This default must be itself an allowed value for this to make sense, so this is checked. This check, however, shouldn't apply to flags that are null by default, since these flags' default value is not parsed by the converter, so there is no guarantee that there exists an accepted user-input value that would also set the value to NULL. In these cases, we assume that "unset" is a distinct value that is always allowed. RELNOTES: None. PiperOrigin-RevId: 187475696
* Warn about config expansions as we do for other expansions.Gravatar ccalvarin2017-12-21
| | | | | | | | | If an expanded value overrides an explicit value, users who do not know the contents of the expansion may be surprised. We already warned about this for hard-coded expansions, and this is now applicable for --config expansions as well. This will only warn when a single-valued option has its value replaced. Options that accumulate multiple values in a list (e.g., --copt) will silently include both explicit and expanded values. RELNOTES: None. PiperOrigin-RevId: 179857526
* Remove wrapper options support.Gravatar ccalvarin2017-12-19
| | | | | RELNOTES: None. PiperOrigin-RevId: 179588174
* Fix canonical option list for options that implicitly require options with ↵Gravatar ccalvarin2017-11-22
| | | | | | | | | allowMultiple=true Was filtering for the implicit options in SingleOptionValue, but forgot the check in RepeatableOptionValue. This is now fixed. RELNOTES: None. PiperOrigin-RevId: 176669853
* Change config expansion application order, gated by startup flag ↵Gravatar ccalvarin2017-11-20
| | | | | | | | | | | | | --expand_configs_in_place. --config options were expanded in a fix-point expansion, where in practice, the flags that --config values expanded to ended up between the normal bazelrc options and the command line's explicit options. Since the options parser has an order-based priority scheme and it accepts multiple mentions of a single-valued option, this conflicts with users' expectations of being able to override these config expansions by using the order in which they are mentioned. This change makes it possible to expand the config values defined in your bazelrc (or blazerc) files to occur in-place: --stuff --config=something --laterstuff will interpret the options that --config=something expands to as if they had been mentioned explicitly between --stuff and --laterstuff. In order to not break users relying on complex flag combinations to configure their builds, this behavior will not yet be turned on by default. Instead, use --expand_configs_in_place as a startup flag to test this feature. --announce_rc may be helpful for debugging any differences between the fixed point and in-place expansions. Once you've debugged your problems, add "startup --expand_configs_in_place" to your blazerc to stick to the new behavior. RELNOTES: Use --expand_configs_in_place as a startup argument to change the order in which --config expansions are interpreted. PiperOrigin-RevId: 176371289
* Cleanup stream use in canonical list of options.Gravatar ccalvarin2017-11-06
| | | | | | | Follow up to https://github.com/bazelbuild/bazel/commit/c50cd13c75a2a1685f5ac9bd70561ac1e50722e7 RELNOTES: None. PiperOrigin-RevId: 174498205
* Remove unused LegacyParamsFilePreProcessor. Removed test cases from ↵Gravatar apell2017-11-03
| | | | | | | OptionsParserTest are implemented in ParamsFilePreProcessorTest, ShellQuotedParamsFilePreProcessorTest and UnquotedParamsFilePreProcessorTest. RELNOTES: None. PiperOrigin-RevId: 174359569
* Switch on tracking of specific option priorities.Gravatar ccalvarin2017-11-03
| | | | | | | Make sure that multiple calls to parse() follow each other sequentially. This is necessary for blazerc expansion, which occurs first in command order, then blazerc order. RELNOTES: None. PiperOrigin-RevId: 174343241
* Compute canonical list of options using OptionValueDescription's tracking of ↵Gravatar ccalvarin2017-10-30
| | | | | | | | | | | | | instances. Stop storing the canonical list of arguments separately. For the canonicalize-flags command, we want to avoid showing options that either have no values in their own right (such as expansion options and wrapper options) and instances of options that did not make it to the final value. This is work we already do in OptionValueDescription, so we can generate the canonical form from the values tracked there, instead of tracking it separately. This means the canonical list is more correct where implicit requirements are concerned: implicit requirements are not listed in the canonical form, but now the values they overwrote will be correctly absent as well. Use this improved list for the effective command line published to the BEP. RELNOTES: Published command lines should have improved lists of effective options. PiperOrigin-RevId: 173873154
* Track expansions in OptionValueDescription.Gravatar ccalvarin2017-10-23
| | | | | | | Add warnings for behavior that is likely unexpected. Expansion values do not accept values at all, and implicit requirements are set regardless of whether the option was turned "on" or not, so warn in the cases where this weird behavior might rear its ugly head. RELNOTES: None PiperOrigin-RevId: 172883214
* Remove feature to allow expansion flags to have values.Gravatar ccalvarin2017-10-18
| | | | | | | | | It was added as a potential fix for --config (an expansion flag with values), but this would have required forcing the parser to know the config's expansions at parsing time, which is not currently possible. Instead, we will use the new addition of option-location tracking to make sure we expand options at a the correct place, even if the expansion is triggered after the fact. This is mostly a straight forward undoing of https://github.com/bazelbuild/bazel/commit/7c7255ec8d6da20526c2c4078c57aadaf3dd3612, except where the context has changed. Notably, implicit requirements are effectively treated like expansion flags, so special casing in OptionDescription could be removed. RELNOTES: None. PiperOrigin-RevId: 172514997
* Migrate all users of OptionsParser.enableParamsFileSupport to use the ↵Gravatar apell2017-10-18
| | | | | | | ShellQuotedParamsFilePreProcessor. This covers all of the tools packaged in the ResourceProcessorBusyBox. RELNOTES: None. PiperOrigin-RevId: 172485486
* Track Option placement within a priority category.Gravatar ccalvarin2017-10-18
| | | | | | | | | An option has precedence over previous options at the same enum-valued priority. Track its placement in this ordering explicitly. This will allow after-the-fact expansion of expansion options such that they correctly take precedence or not compared to other mentions of the same flag. This is needed to fix --config's expansion. RELNOTES: None. PiperOrigin-RevId: 172367996
* Make option conflicts less spammy.Gravatar ccalvarin2017-10-16
| | | | | | | Remove an unnecessary warning and make all warnings for option conflicts print only if the option values are not equal. RELNOTES: None. PiperOrigin-RevId: 172124261
* Expand implicitRequirements in the location of the option that required it.Gravatar ccalvarin2017-10-13
| | | | | | | | | | | Removes the special casing of implicit requirements. Accumulating them and parsing them at the end of the parse() function was never enough to actually guarantee that the value not be replaced. I've gone through all options with implicit requirements to make sure that the expectation is checked after options parsing, so this change should be relatively safe. Implicit requirements is still a broken concept - they don't actually expand based on the value given, so a user that is explicitly NOT setting a flag might unwittingly be setting all the requirements for that unset flag. Removing it fully requires redesigning or removing the flags that set it, though, so for now we are standardizing the behavior so that it behaves like any other expansion options, just one with a value. Also consolidate the deprecated wrapper option behavior into the expansion work. It will soon be removed entirely, but for now it can get grouped in with the expansion logic, so that its differences are more explicit. RELNOTES: None. PiperOrigin-RevId: 171957502
* Clean up InvocationPolicy's use of OptionDescription.Gravatar ccalvarin2017-10-11
| | | | | | OptionDescription is basically a hack to get the expansion data for options from outside the options parser, but it was being used at various points of invocation policy enforcement. In order to correctly track option origin, we only want to get this information once. Do it during the invocation policy expansion stage, not at enforcement, so that we track the information of the option's origin in the original invocation policy passed to the enforcer, not the expanded one. PiperOrigin-RevId: 171661669
* Report the structured Bazel command line via the BEP.Gravatar ccalvarin2017-10-10
| | | | | | | | | This is part of the effort outlined in https://bazel.build/designs/2017/07/13/improved-command-line-reporting.html. The refactoring of the options parser is not yet complete, so we still do not have complete & correct information about the canonical command line. Where the information is blatantly incorrect, a best approximation was made, with comments and tests documenting the deficiencies. Change the names of the initial CommandLine fields in the BEP to be explicitly identified as unstructured. RELNOTES: None. PiperOrigin-RevId: 171625377
* Downgrade the default invocation policy log levels to fine.Gravatar ccalvarin2017-10-06
| | | | | | | | Unfortunately, since the invocation policy gets enforced on a user's command line early in the command environment setup, so the effective log level is not yet set. For this run, keep the logs at INFO, since any other level will disappear into the ether. InvocationPolicy gets enforced not only at invocation startup but also elsewhere to recompute the default values. The log statements from these different runs overwhelm the log output from our tests, making it hard to find other events without filtering the log. Make all extra invocation policy enforcements log at FINE so that they only appear when extra detail is requested. PiperOrigin-RevId: 171151573
* Categorize build options for BuildConfiguration.Gravatar gregce2017-10-06
| | | | PiperOrigin-RevId: 171017483
* Add new option categorization and tagging information to HelpCommand's output.Gravatar ccalvarin2017-09-27
| | | | | | | | | | | | | If setting flag --use_new_category_enum, group the options by the new categories in both the command line output and the "everything-as-html" output used for the generated docs at https://bazel.build/versions/master/docs/command-line-reference.html. In the html output, the effect and metadata tags are listed for each option, with links to their descriptions at the bottom of the page. The tags only appear in the terminal output in -l/--long/--help_verbosity=long, and only the names appear. This is still experimental as the majority of options do not yet use the new categorization system. The new output can be seen in command-line-reference.html format by adding the new flag to the "help everything-as-html" line in //src/main/java/com/google/devtools/build/lib:gen_command-line-reference. The html output is in the same order as before (by blaze rule, with inherited options not repeated), which means it still has duplicate options, that should ideally be fixed separately. I propose either dropping the high-level grouping and just grouping the options by documentation category, or potentially grouping them by optionsbase in some non-class-naming way, and listing the commands that they apply to, as more and more optionsbases are used by multiple commands without being inherited (for example, all BuildEventServiceOptions are listed 20 times). People probably use ctrl-f to navigate this page anyway. Once we know that we only list each option once, we can actually have links to the options, which will make it possible to have links in the expansion lists. Issue #3758 RELNOTES: added experimental --use_new_category_enum to the help command to output options grouped by the new type of category. PiperOrigin-RevId: 170116553
* Move the canonicalization of an option value to the option value itself.Gravatar ccalvarin2017-09-27
| | | | | | | Ideally, the canonical form we output from OptionUtils would be the same as for the command canonicalize-flags, but that must wait for dependencies to be cleaned up. Still, in the meantime, keep the --foo=1 normalization of --foo, and apply it to all other boolean flag values (t,true,yes,y, and false equivalents), so that the canoncalize-flags output is more canonical, even if it isn't using the --[no]foo form yet. RELNOTES: Boolean flag values will now get normalized to 1 or 0 in canonicalize-flags output. PiperOrigin-RevId: 170084599
* Track the origin of an option in the option instance, not its final value.Gravatar ccalvarin2017-09-21
| | | | | | | | | A single instance of an option has a single origin, but the final value only has a single origin if it has a single value. For multi-valued options, it is wrong to expect that the final value of an option will have a single parent. Track the option parents (which option expanded to the current instance, if any) in the right place, with the ParsedOptionDescription. Also fix some inconsistent spelling of 'dependent,' in favor of the American English standard. RELNOTES: None. PiperOrigin-RevId: 169487515
* Deprecate wrapperOptions.Gravatar ccalvarin2017-09-21
| | | | | | | In order to discourage new uses (there shouldn't be any, but just in case), make it illegal to set wrapperOption=true for non deprecated options. RELNOTES: None. PiperOrigin-RevId: 169477990
* Track all option instances.Gravatar ccalvarin2017-09-21
| | | | | | | Regardless of option type, have the OptionValueDescription track information about the parsed option. This does not yet change the values that we output as part of the GotOptionsEvent, as the expansion, implicit req, and wrapper options still don't actually track anything in their option value. RELNOTES: None. PiperOrigin-RevId: 169469793
* Cleanup of expansion option namingGravatar ccalvarin2017-09-21
| | | | | | | | | Options that expand to other options are expansion options and the options they expand to have values that were expansions. This can be a bit confusing. Removes the isExpansion() call that is somewhat ambiguous, and forces option users to explicitly check the option definition for this information. Also provide a parallel boolean function for implicit requirements, so that we stop querying for the length of the implicit requirement all over the place. RELNOTES: None. PiperOrigin-RevId: 169461566
* Create multiple ParamsFilePreProcessors to allow parsing files using the ↵Gravatar apell2017-09-21
| | | | | | | formats specified in ParameterFile.ParameterFileType. Also maintain the currently used parsing style of whitespace split arguments that allows single and double quoting and whitespace and quote escaping. This style of parsing is for a format not currently generated and will be removed once all consuming actions have been converted to using a supported format explicitly. RELNOTES: None. PiperOrigin-RevId: 169437362
* Document the options in smaller commands.Gravatar ccalvarin2017-09-20
| | | | | RELNOTES: None. PiperOrigin-RevId: 169258065
* Treat parsed option values differently by option type.Gravatar ccalvarin2017-09-19
| | | | | | | | | | | | | | | | There is a vexingly large world of possible option types, each with its own quirks of how it interfaces with new inputs as they come in: values can be - overridden (default) - concatenated (allowMultiple) - flattened (allowMultiple that accepts list inputs) - disappear into additional flag inputs (expansion flags) Or some combination of the above, in the case of flags with implicit dependencies and wrapper options. Begin removing the error-prone treatment of all option types with conditional branches. This model of the different options will make it much easier to isolate the option-type specific logic with the command-line parsing logic. Flags that affect other flags (implicit requirements, expansions, and wrappers) will be migrated in a later change. This CL does not change flag parsing semantics, just migrates the current parsing logic to the new class structure. RELNOTES: None. PiperOrigin-RevId: 169239182
* Throw away the confusing UnparsedOptionValueDescription name.Gravatar ccalvarin2017-09-18
| | | | | | | | | | | | | | We get UnparsedValues after ... parsing the options. So that doesn't make sense. What was meant was that it wasn't converted to the final value. In an effort to make this distinction more clear, this change will make the terminology more consistent. The `--foo=bar` step is "parsing" and the `bar -> Object` step is "converting" (it is, in fact, done by Converters). RELNOTES: None. PiperOrigin-RevId: 168852847
* Introduce "bazel help flags-as-proto" to print information about supported ↵Gravatar fwe2017-09-14
| | | | | | flags as a text protobuf. PiperOrigin-RevId: 168695297
* Pass the UnparsedOptionValues when setting or adding option values.Gravatar ccalvarin2017-09-14
| | | | | | | | Stop breaking the value apart to be recombined later. Also stop using OptionValueDescriptions as though we have a final option when expanding flags for invocation policy. These values are explicitly the output from parsing the expansion strings, not yet converted or combined with other values of the same flag. After this change, only UnparsedOptionValueDescription should be used when strings of flags are parsed, and OptionValueDescription should be used when the final version of a flag is created or used. PiperOrigin-RevId: 168688063
* Consolidate the "unparsed" option value tracking.Gravatar ccalvarin2017-09-14
| | | | | | | In preparation for linking the parsed and unparsed values of options, consolidate and standardize our representation of the flag values as we received them (what is meant by "unparsed" values in this case). This was being done separately in ParseOptionResult, which, with extra context added, is being folded into UnparsedOptionValueDescription. We now track how an option was provided and where it came from for all option parsing. RELNOTES: None. PiperOrigin-RevId: 168682082
* Move option value tracking classes to their own file.Gravatar ccalvarin2017-09-13
| | | | | | | These classes are mostly used during the options parsing process itself, and are barely a part of the options parser interface, so they really don't belong in OptionsParser.java. They are also about to change significantly, so taking this opportunity to split them out. RELNOTES: None. PiperOrigin-RevId: 168400162
* Replace referrals to options by their name to option definitions.Gravatar ccalvarin2017-09-12
| | | | | | | Now that we have a standard way of referring to an option, remove all of the places that we were referring to them by their name. Since options can have multiple names, this is more clear and provides the additional information needed to understand the option. It also stops the habit of requesting unqualified strings, which was hard to read. RELNOTES: None. PiperOrigin-RevId: 168254584
* Option value tracking should refer to the option definitions, not just track ↵Gravatar ccalvarin2017-09-11
| | | | | | | | | | | option name. UnparsedOptionValueDescription and OptionValueDescription both had redundant information about options, since they tracked options by their names, and not their definition. Remove the redundancy. For getEffectiveOptions, the old behavior was relying on the fact that all options were given allowmulitple=false, which was wrong, so stop passing in the default value where it would get confused with values that have been created and tracked by the options parser. Instead, passing value=null is the way to indicate that an option is unset. RELNOTES: None. PiperOrigin-RevId: 168038067
* Options with oldNames will no longer get reported twice in the effective ↵Gravatar ccalvarin2017-09-11
| | | | | | | | | | option lists. Tracking the names together for option identification was useful, but then the same list was being used as the source of options for the parser, which lead to some options being listed twice. Also complete a few tests that should have already been tested in different orders. PiperOrigin-RevId: 168024719
* More BUILD file refactorings.Gravatar philwo2017-09-06
| | | | | | | | | Split collect, concurrent, vfs, windows into package-level BUILD files. Move clock classes out of "util", into their own Java package. Move CompactHashSet into its own Java package to break a dependency cycle. Give nestedset and inmemoryfs their own package-level BUILD files. PiperOrigin-RevId: 167702127
* Expansion flags need some restraints.Gravatar ccalvarin2017-09-06
| | | | | | | | How expanding flags interact with other possible flag qualities is not defined. Should repeated values have effects multiple times and accumulate? This doesn't really make sense, expansion flags don't have values that would accumulate. For this reason, don't allow expansion options to have allowMultiple set to true. Likewise for other behaviors. PiperOrigin-RevId: 167580641
* Rename all logger instances to "logger" (instead "LOG" or "log").Gravatar lberki2017-09-05
| | | | | RELNOTES: None. PiperOrigin-RevId: 167505493