// Copyright 2018 The Bazel Authors. All rights reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package com.google.devtools.build.lib.skylarkbuildapi; import com.google.devtools.build.lib.events.Location; import com.google.devtools.build.lib.skylarkinterface.Param; import com.google.devtools.build.lib.skylarkinterface.ParamType; import com.google.devtools.build.lib.skylarkinterface.SkylarkCallable; import com.google.devtools.build.lib.skylarkinterface.SkylarkModule; import com.google.devtools.build.lib.skylarkinterface.SkylarkModuleCategory; import com.google.devtools.build.lib.skylarkinterface.SkylarkValue; import com.google.devtools.build.lib.syntax.BaseFunction; import com.google.devtools.build.lib.syntax.EvalException; import com.google.devtools.build.lib.syntax.Runtime; import com.google.devtools.build.lib.syntax.SkylarkList; import com.google.devtools.build.lib.syntax.SkylarkNestedSet; /** Command line args module. */ @SkylarkModule( name = "Args", category = SkylarkModuleCategory.BUILTIN, doc = "An object that encapsulates, in a memory-efficient way, the data needed to build part or " + "all of a command line." + "" + "

It often happens that an action requires a large command line containing values " + "accumulated from transitive dependencies. For example, a linker command line might " + "list every object file needed by all of the libraries being linked. It is best " + "practice to store such transitive data in depset" + "s, so that they can be shared by multiple targets. However, if the rule " + "author had to convert these depsets into lists of strings in order to construct an " + "action command line, it would defeat this memory-sharing optimization." + "" + "

For this reason, the action-constructing functions accept Args " + "objects in addition to strings. Each Args object represents a " + "concatenation of strings and depsets, with optional transformations for " + "manipulating the data. Args objects do not process the depsets they " + "encapsulate until the execution phase, when it comes time to calculate the command " + "line. This helps defer any expensive copying until after the analysis phase is " + "complete. See the Optimizing Performance page " + "for more information." + "" + "

Args are constructed by calling " + "ctx.actions.args(). They can be passed as the arguments " + "parameter of ctx.actions.run() or " + "ctx.actions.run_shell(). Each " + "mutation of an Args object appends values to the eventual command " + "line." + "" + "

The map_each feature allows you to customize how items are " + "transformed into strings. If you do not provide a map_each function, " + "the standard conversion is as follows: " + "

" + "" + "

When using string formatting (format, format_each, and " + "format_joined params of the add*() methods), the format " + "template is interpreted in the same way as %-substitution on strings, " + "except that the template must have exactly one substitution placeholder and it must " + "be %s. Literal percents may be escaped as %%. Formatting " + "is applied after the value is converted to a string as per the above." + "" + "

Each of the add*() methods have an alternate form that accepts an " + "extra positional parameter, an \"arg name\" string to insert before the rest of the " + "arguments. For add_all and add_joined the extra string " + "will not be added if the sequence turns out to be empty. " + "For instance, the same usage can add either --foo val1 val2 val3 --bar" + " or just --bar to the command line, depending on whether the " + "given sequence contains val1..val3 or is empty." + "" + "

If the size of the command line can grow longer than the maximum size allowed by " + "the system, the arguments can be spilled over into parameter files. See " + "use_param_file() and " + "set_param_file_format()." + "" + "

Example: Suppose we wanted to generate the command line: " + "

\n"
          + "--foo foo1.txt foo2.txt ... fooN.txt --bar bar1.txt,bar2.txt,...,barM.txt --baz\n"
          + "
" + "We could use the following Args object: " + "
\n"
          + "# foo_deps and bar_deps are depsets containing\n"
          + "# File objects for the foo and bar .txt files.\n"
          + "args = ctx.actions.args()\n"
          + "args.add_all(\"--foo\", foo_deps)\n"
          + "args.add_joined(\"--bar\", bar_deps, join_with=\",\")\n"
          + "args.add(\"--baz\")\n"
          + "ctx.actions.run(\n"
          + "  ...\n"
          + "  arguments = [args],\n"
          + "  ...\n"
          + ")\n"
          + "
" ) public interface CommandLineArgsApi extends SkylarkValue { @SkylarkCallable( name = "add", doc = "Appends an argument to this command line." + "" + "

Deprecation note: The before_each, join_with " + "and map_fn params are replaced by the " + "add_all() and add_joined() " + "methods. These parameters will be removed, and are currently disallowed if the " + "" + "--incompatible_disallow_old_style_args_add flag is set. Likewise, " + "value should now be a scalar value, not a list, tuple, or depset of " + "items.", parameters = { @Param( name = "arg_name_or_value", doc = "If two positional parameters are passed this is interpreted as the arg name. " + "The arg name is added before the value without any processing. " + "If only one positional parameter is passed, it is interpreted as " + "value (see below)."), @Param( name = "value", defaultValue = "unbound", doc = "The object to append. It will be converted to a string using the standard " + "conversion mentioned above. Since there is no map_each parameter " + "for this function, value should be either a string or a " + "File." + "" + "

Deprecated behavior: value may also be a list, tuple, " + "or depset of multiple items to append."), @Param( name = "format", type = String.class, named = true, positional = false, defaultValue = "None", noneable = true, doc = "A format string pattern, to be applied to the stringified version of value" + "." + "" + "

Deprecated behavior: If value is a list or depset, " + "formatting is applied to each item."), @Param( name = "before_each", type = String.class, named = true, positional = false, defaultValue = "None", noneable = true, doc = "Deprecated: Only supported when value is a list, tuple, or " + "depset. This string will be appended prior to appending each item."), @Param( name = "join_with", type = String.class, named = true, positional = false, defaultValue = "None", noneable = true, doc = "Deprecated: Only supported when value is a list, tuple, or " + "depset. All items will be joined together using this string to form a single " + "arg to append."), @Param( name = "map_fn", type = BaseFunction.class, named = true, positional = false, defaultValue = "None", noneable = true, doc = "Deprecated: Only supported when value is a list, tuple, or " + "depset. This is a function that transforms the sequence of items into a list " + "of strings. The sequence of items is given as a positional argument -- the " + "function must not take any other parameters -- and the returned list's length " + "must equal the number of items. Use map_each of add_all" + " or add_joined instead.") }, useLocation = true ) public Runtime.NoneType addArgument( Object argNameOrValue, Object value, Object format, Object beforeEach, Object joinWith, Object mapFn, Location loc) throws EvalException; @SkylarkCallable( name = "add_all", doc = "Appends multiple arguments to this command line. For depsets, the items are " + "evaluated lazily during the execution phase." + "" + "

Most of the processing occurs over a list of arguments to be appended, as per " + "the following steps:" + "

    " + "
  1. If map_each is given, it is applied to each input item, and the " + " resulting lists of strings are concatenated to form the initial argument " + " list. Otherwise, the initial argument list is the result of applying the " + " standard conversion to each item." + "
  2. Each argument in the list is formatted with format_each, if " + " present." + "
  3. If uniquify is true, duplicate arguments are removed. The first " + " occurrence is the one that remains." + "
  4. If a before_each string is given, it is inserted as a new " + " argument before each existing argument in the list. This effectively doubles " + " the number of arguments to be appended by this point." + "
  5. Except in the case that the list is empty and omit_if_empty is " + " true (the default), the arg name and terminate_with are " + " inserted as the first and last arguments, respectively, if they are given." + "
" + "Note that empty strings are valid arguments that are subject to all these " + "processing steps.", parameters = { @Param( name = "arg_name_or_values", doc = "If two positional parameters are passed this is interpreted as the arg name. " + "The arg name is added before the values without any processing. " + "This arg name will not be added if omit_if_empty is true " + "(the default) and no other items are appended (as happens if " + "values is empty or all of its items are filtered). " + "If only one positional parameter is passed, it is interpreted as " + "values (see below)." ), @Param( name = "values", allowedTypes = { @ParamType(type = SkylarkList.class), @ParamType(type = SkylarkNestedSet.class), }, defaultValue = "unbound", doc = "The list, tuple, or depset whose items will be appended." ), @Param( name = "map_each", type = BaseFunction.class, named = true, positional = false, defaultValue = "None", noneable = true, doc = "A function that converts each item to zero or more strings, which may be further " + "processed before appending. If this param is not provided, the standard " + "conversion is used." + "" + "

The function takes in the item as a positional parameter and must have no " + "other parameters. The return value's type depends on how many arguments " + "are to be produced for the item:" + "

" + "Returning a single string or None has the same effect as " + "returning a list of length 1 or length 0 respectively. However, it is more " + "efficient and readable to avoid creating a list where it is not needed." + "" + "

Warning: print() " + "statements that are executed during the call to map_each will " + "not produce any visible output." ), @Param( name = "format_each", type = String.class, named = true, positional = false, defaultValue = "None", noneable = true, doc = "An optional format string pattern, applied to each string returned by the " + "map_each function. " + "The format string must have exactly one '%s' placeholder." ), @Param( name = "before_each", type = String.class, named = true, positional = false, defaultValue = "None", noneable = true, doc = "An optional string to append before each argument derived from values " + "is appended." ), @Param( name = "omit_if_empty", type = Boolean.class, named = true, positional = false, defaultValue = "True", doc = "If true, if there are no arguments derived from values to be " + "appended, then all further processing is suppressed and the command line will " + "be unchanged. If false, the arg name and terminate_with, " + "if provided, will still be appended regardless of whether or not there are " + "other arguments." ), @Param( name = "uniquify", type = Boolean.class, named = true, positional = false, defaultValue = "False", doc = "If true, duplicate arguments that are derived from values will be " + "omitted. Only the first occurrence of each argument will remain. Usually this " + "feature is not needed because depsets already omit duplicates, but it can be " + "useful if map_each emits the same string for multiple items." ), @Param( name = "terminate_with", type = String.class, named = true, positional = false, defaultValue = "None", noneable = true, doc = "An optional string to append after all other arguments. This string will not be " + "added if omit_if_empty is true (the default) and no other items " + "are appended (as happens if values is empty or all of its items " + "are filtered)." ), }, useLocation = true ) public Runtime.NoneType addAll( Object argNameOrValue, Object values, Object mapEach, Object formatEach, Object beforeEach, Boolean omitIfEmpty, Boolean uniquify, Object terminateWith, Location loc) throws EvalException; @SkylarkCallable( name = "add_joined", doc = "Appends an argument to this command line by concatenating together multiple values " + "using a separator. For depsets, the items are evaluated lazily during the " + "execution phase." + "" + "

Processing is similar to add_all(), but " + "the list of arguments derived from values is combined into a single " + "argument as if by join_with.join(...), and then formatted using the " + "given format_joined string template. Unlike add_all(), " + "there is no before_each or terminate_with parameter " + "since these are not generally useful when the items are combined into a single " + "argument." + "" + "

If after filtering there are no strings to join into an argument, and if " + "omit_if_empty is true (the default), no processing is done. " + "Otherwise if there are no strings to join but omit_if_empty is " + "false, the joined string will be an empty string.", parameters = { @Param( name = "arg_name_or_values", doc = "If two positional parameters are passed this is interpreted as the arg name. " + "The arg name is added before values without any processing. " + "This arg will not be added if omit_if_empty is true " + "(the default) and there are no strings derived from values " + "to join together (which can happen if values is empty " + "or all of its items are filtered)." + "If only one positional parameter is passed, it is interpreted as " + "values (see below)."), @Param( name = "values", allowedTypes = { @ParamType(type = SkylarkList.class), @ParamType(type = SkylarkNestedSet.class), }, defaultValue = "unbound", doc = "The list, tuple, or depset whose items will be joined."), @Param( name = "join_with", type = String.class, named = true, positional = false, doc = "A delimiter string used to join together the strings obtained from applying " + "map_each and format_each, in the same manner as " + "string.join()."), @Param( name = "map_each", type = BaseFunction.class, named = true, positional = false, defaultValue = "None", noneable = true, doc = "Same as for add_all."), @Param( name = "format_each", type = String.class, named = true, positional = false, defaultValue = "None", noneable = true, doc = "Same as for add_all." ), @Param( name = "format_joined", type = String.class, named = true, positional = false, defaultValue = "None", noneable = true, doc = "An optional format string pattern applied to the joined string. " + "The format string must have exactly one '%s' placeholder." ), @Param( name = "omit_if_empty", type = Boolean.class, named = true, positional = false, defaultValue = "True", doc = "If true, if there are no strings to join together (either because values" + " is empty or all its items are filtered), then all further processing " + "is suppressed and the command line will be unchanged. If false, then even if " + "there are no strings to join together, two arguments will be appended: " + "the arg name followed by an empty string (which is the logical join " + "of zero strings)." ), @Param( name = "uniquify", type = Boolean.class, named = true, positional = false, defaultValue = "False", doc = "Same as for add_all." ) }, useLocation = true ) public Runtime.NoneType addJoined( Object argNameOrValue, Object values, String joinWith, Object mapEach, Object formatEach, Object formatJoined, Boolean omitIfEmpty, Boolean uniquify, Location loc) throws EvalException; @SkylarkCallable( name = "use_param_file", doc = "Spills the args to a params file, replacing them with a pointer to the param file. " + "Use when your args may be too large for the system's command length limits ", parameters = { @Param( name = "param_file_arg", type = String.class, named = true, doc = "A format string with a single \"%s\". " + "If the args are spilled to a params file then they are replaced " + "with an argument consisting of this string formatted with " + "the path of the params file."), @Param( name = "use_always", type = Boolean.class, named = true, positional = false, defaultValue = "False", doc = "Whether to always spill the args to a params file. If false, " + "bazel will decide whether the arguments need to be spilled " + "based on your system and arg length.") }) public void useParamsFile(String paramFileArg, Boolean useAlways) throws EvalException; @SkylarkCallable( name = "set_param_file_format", doc = "Sets the format of the param file when written to disk", parameters = { @Param( name = "format", type = String.class, named = true, doc = "The format of the param file. Must be one of:

" + "

The format defaults to \"shell\" if not called.") }) public void setParamFileFormat(String format) throws EvalException; }