diff options
author | Vladimir Moskva <vladmos@google.com> | 2016-08-05 16:48:07 +0000 |
---|---|---|
committer | Yue Gan <yueg@google.com> | 2016-08-08 08:06:54 +0000 |
commit | ea381f0c8a103936e00a07b79a6ca8f8c81f2819 (patch) | |
tree | c05197b341e988474766b6d2d9b2f43d38c17c71 /site/designs/skylark/declared-providers.md | |
parent | d313a4ec65a85e3aa8421c609f71a116ae4fc64d (diff) |
Added a design document and fixed links
--
Change-Id: I1a904d19e1a63eff8c245bac3c18eb4ce9838912
Reviewed-on: https://bazel-review.googlesource.com/#/c/4271/
MOS_MIGRATED_REVID=129451413
Diffstat (limited to 'site/designs/skylark/declared-providers.md')
-rw-r--r-- | site/designs/skylark/declared-providers.md | 284 |
1 files changed, 284 insertions, 0 deletions
diff --git a/site/designs/skylark/declared-providers.md b/site/designs/skylark/declared-providers.md new file mode 100644 index 0000000000..51aab95a93 --- /dev/null +++ b/site/designs/skylark/declared-providers.md @@ -0,0 +1,284 @@ +--- +layout: documentation +title: Declared Providers +--- + +# Declared Providers + +Authors: [Dmitry Lomov](mailto:dslomov@google.com), +[Laurent Le Brun](mailto:laurentlb@google.com) + +Status: Approved + +Date: 2016-06-06 + +## Motivation + +Skylark rules use simple Skylark structs as their providers. Skylark providers +are identified as simple names, such as 'java' or 'files'. This approach has the +advantage of simplicity, but as the number and complexity of Skylark rules grow, +we run into engineering scalability problems: + +* Using simple names for providers might lead to name conflicts (when + unrelated rules call their providers the same simple name). +* There is no clear formal way to add documentation for those providers; if + any, the documentation is in prose in rule's doc comment, where it tends to + become obsolete/incomplete; most existing providers have no documentation + explaining their contracts at all. +* It’s hard to know which fields to expect in a provider. +* It’s hard to know which rule can depend on which rule. + +## Goals + +* Solve name-conflict problem for providers +* Allow to specify providers in Skylark rules with the same level of + robustness as other components of the language, such as rules and aspects +* Enable the same or better documentability of Skylark providers as native + providers allow +* Improve providers interoperability with native code. + +## Proposal + +We propose a redesign of how Skylark rules deal with providers to address the +above concerns. The redesign can occur in stages; those stages represent +implementation stages, but allow Skylark users to gradually opt for "more and +more engineering" as their custom rules progress from a small project on the +side to a public release. + +Our proposal is backwards compatible with the existing providers in Bazel and +allows easy, gradual piecemeal replacement of them. + +### Stage 1: Solving the Naming Problem + +Under the new proposal, a minimum implementation of a custom provider looks like +this: + +``` +# rust.bzl + +# Introduces a provider. `rust_provider` is now both a function +# that can be used to construct the provider instance, +# and a "symbol" that can be used to access it. +rust_provider = provider() + +def _impl(ctx): + # `rust_provider` is used as struct-like constructor + # it accepts the same arguments as a standard `struct` function + rust = rust_provider(defines = "-DFOO", ...) + # return value of rule implementation function + # is just a list of providers; their "names" are specified + # by their constructors, see below + return [ctx.provider(files = ...), rust] +rust_library = rule(implementation = _impl, + # Optional declaration; the rule MUST provide all the + # providers in this list + providers = [rust_provider]) +``` + +``` +# Example of how to access the provider + +load(":rust.bzl", "rust_provider") + +def _impl(ctx): + dep = ctx.attr.deps[0] # Target object + # `rust_provider` is used as a key to access a particular + # provider + defines = dep[rust_provider].defines ... +``` + +#### The provider function + +* We introduce two new kinds of Skylark values, a *provider declaration* and a + *provider*. +* Provider declaration (`rust_provider` in the example) is created using the + `provider` function. +* Provider declaration can be used to construct a *provider* (`rust` in the + example). Provider is a struct-like Skylark value, with the only difference + that every provider is associated with its declaration (it is a different + type). Arguments of a provider declaration when used as a function are + exactly the same as that of a built-in `struct` function. +* [Target](http://www.bazel.io/docs/skylark/lib/Target.html) objects become + dictionaries of providers indexed by their declaration. Bracket notation can + be used to retrieve a particular provider. Thus, provider declarations are + [symbol-like](https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Symbol) + values. +* Providers can be private to an extension file; in that case the provider + cannot be accessed outside that file. + +#### Default providers (ctx.provider) + +There is a set of default providers (`files`, `runfiles`, `data_runfiles`, +`executable`, `output_groups`, etc.). We group them in a single provider, +`ctx.provider`: + +``` +defaults = ctx.provider(files = set(), runfiles = ...) +``` + +The current set of APIs on Target objects that access these providers +(`target.files`, `target.output_group("name")` etc.) will continue to work. + +#### Return value + +The implementation function can return either a provider, or a list of +providers. It is an error to return two providers of the same type. + +``` +return [defaults, rust, cc] +return ctx.provider(files = set()) +``` + +#### Declaring providers returned by a rule + +Users need to know which rules provide which providers. This is important for +documentation and for knowing which dependencies are allowed (e.g. we want to +find easily what can go in the deps attribute of cc_library). + +We allow rules to declare the providers they intend to return with a `providers` +argument of a +<code>[rule](http://www.bazel.io/docs/skylark/lib/globals.html#rule)</code> +function. It is an error if the rule implementation function does not return all +the providers listed in `providers`. It may however return additional providers. + +``` +rust_provider = provider() + +rust_library = rule(implementation = _impl, + # Optional declaration; the rule MUST provide all the + # providers in this list + providers = [rust_provider]) +``` + +#### Migration path and support for "legacy" providers + +To support current model of returning providers, where they are identified by a +simple name, we continue to allow providers name in the return struct: + +``` +def _impl(ctx): + ... + return struct( + legacy_provider = struct(...), + files = set(...), + providers = [rust]) +``` + +This also works for “default” providers, such as “files”, “runfiles” etc. +However if one of those legacy names is specified, it is an error to have +ctx.provider instance in the list of `providers`. + +We also allow returning a declared provider both directly and with a simple +name: + +``` +def _impl(ctx): + ... + return struct(rust = rust, providers = [rust]) +``` + +This allows the rules to mix old and new style, and migrate rule definition to a +new style without changing all the uses of that rule. + +Old-style providers with simple names can still be accessed with dot-notation on +Target object, so all of the following is valid. + +Old-style usage: + +* `target.rust` (=> rust) +* `getattr(target, "rust")` (=> rust) +* `hasattr(target, "rust")` (=> True) + +New-style usage: + +* `target[rust_provider]` (=> rust) +* `rust_provider in target` (=> True) +* `target.keys` (=> [rust_provider]) +* `target.values` (=> [rust]) +* `target.items` (=> [(rust_provider, rust)]) + +#### type function + +Type function on providers returns a string `"provider"`. Type function on a +provider instance returns a string `"struct"`. + +### Stage 2: Documentation and Fields + +Provider declarations are a convenient place to add more annotations to +providers. We propose 2 specific things there: + +``` +rust_provider = provider( + doc = "This provider contains Rust information ...", + fields = ["defines", "transitive_deps"] +) +``` + +This specifies documentation for the provider and a list of fields that the +provider can have. + +If `fields` argument is present, extra, undeclared fields are not allowed. + +Both `doc` and `fields` arguments to `provider` function are optional. + +`fields` argument can also be a dictionary (from string to string), in that case +the keys are names of fields, and the values are documentation strings about +individual fields + +``` +rust_provider = provider( + doc = "This provider contains Rust information ...", + fields = { + "defines": "doc for define", + "transitive_deps": "doc for transitive deps, + }) +``` + +### Native Providers + +Providers (as Skylark values) can be also declared natively. A set of +annotations can be developed to facilitate declaring them with little effort. + +As a strawman example: + +``` +/** + * Hypothetical implementation of Skylark provider value (result of + * provider(..) function. + */ +class SkylarkProviderValue extends SkylarkValue { + ... + /** + * Creates a SkylarkProviderValue for a native provider + * `native` must be annotated with @SkylarkProvider annotation. + * Field accessors and constructor function appear magically. + */ + static <T> SkylarkProviderValue forNative(Class<T> native) { ... } +} + +@SkylarkProvider(builder = Builder.class) +// A class with this annotation can be used as provider declaration +class rustProvider implements TransitiveInfoProvider { + @SkylarkProviderField(doc = ...) + // Skylark name is 'defines' + String getDefines() { ... } + + @SkylarkProviderField(doc = ...) + // Skylark name is 'transitive_deps' + NestedSet<Artifact> getTransitiveDeps() { ... } + + @SkylarkProviderField(doc = ...) + // Not allowed, the set of types exposed to Skylark is restricted + DottedVersion getVersion() { ... } + + // Automatically used to provide an implementation for + // construction function. + static class Builder { + // a setter for 'defines' field, based on name. + void setDefines(String defines) { ... } + // a setter for 'transitive_deps' field, based on name. + void setTransitiveDeps(...) {...} + rustProvider build() { ... } + } +} +``` |