diff options
author | Damien Martin-Guillerez <dmarting@google.com> | 2016-08-02 09:44:34 +0000 |
---|---|---|
committer | Philipp Wollermann <philwo@google.com> | 2016-08-02 11:16:24 +0000 |
commit | 2a8d73109f5e539fc20e956e8139dc1bb5b65111 (patch) | |
tree | 26a2ba8a328e784ffe226c4e3bf89ae251d1eafb /site/designs/_posts | |
parent | 5592a0aae2bebfbea278fdf8125bf26831a65975 (diff) |
Add Skylark Remote Repositories design docs to the website.
The original docs are at
bazel init (removed doc)
Skylark Remote Repositories: https://docs.google.com/document/d/1jKbNXOVp2T1zJD_iRnVr8k5D0xZKgO8blMVDlXOksJg/
Generating C++ crosstool with a Skylark Remote Repository: https://docs.google.com/document/d/1Ju74cmBCfYl8SEZT_tXhRmbi49ZLMzXgoQzzqve-dxI
--
Change-Id: I5acb28c08c0de552c259e091249e8787c2cf0e58
Reviewed-on: https://bazel-review.googlesource.com/4222
MOS_MIGRATED_REVID=129078523
Diffstat (limited to 'site/designs/_posts')
-rw-r--r-- | site/designs/_posts/2015-03-06-bazel-init.md | 298 | ||||
-rw-r--r-- | site/designs/_posts/2015-07-02-skylark-remote-repositories.md | 161 | ||||
-rw-r--r-- | site/designs/_posts/2016-02-16-cpp-autoconf.md | 145 |
3 files changed, 604 insertions, 0 deletions
diff --git a/site/designs/_posts/2015-03-06-bazel-init.md b/site/designs/_posts/2015-03-06-bazel-init.md new file mode 100644 index 0000000000..faf0c0934f --- /dev/null +++ b/site/designs/_posts/2015-03-06-bazel-init.md @@ -0,0 +1,298 @@ +--- +layout: contribute +title: bazel init a.k.a ./configure for Bazel +--- + +__This design document has be replaced by +[Skylark Remote Repositories](/designs/2015/07/02/skylark-remote-repositories.html) +and is maintained here just for reference__ + +# bazel init a.k.a ./configure for Bazel +_A configuration mechanism for Bazel_ + +__Status:__ deprecated, replaced by [Skylark Remote Repositories](/designs/2015/07/02/skylark-remote-repositories.html) + +__Author:__ dmarting@google.com + +## I. Rationale + +Bazel tooling needs special setup to work. For example, C++ crosstool +configuration requires path to GCC or Java configuration requires the +path to the JDK. Autodetecting those paths from Bazel would be broken +because each ruleset requires its own configuration (C++ CROSSTOOL +information is totally different from JDK detection or from go root +detection). Therefore, providing a general mechanism to configure +Bazel tooling seems natural. To have Bazel self-contained, we will +ship this mechanism as an additional command of Bazel. Because this +command deals with non-hermetic parts of Bazel, this command should +also group all non-hermetic steps (i.e. it should fetch the +dependencies from the remote repositories) so a user can run it and +get on a plane with everything needed. + +## II. Considered use-cases + +We consider the 3 following use-cases: + + - __UC1.__ The user wants to not worry about tools configuration and + use the default one for golden languages (Java, C++, Shell) and + wants to also activate an optional language (Go). No configuration + information (aka `tools` package) should be checked into the + version control system. + - __UC2.__ The user wants to tweak Java configuration but not C++. Of + course, the user wants his tweak to be shared with his team (i.e. + `tools/jdk` should be checked into the version control system). + However, the user does not want to have C++ information (i.e. + `tools/cpp`) in the VCS. + - __UC3.__ The user wants his build to be hermetic and he wants to + set up everything in his `tools` directory (Google use-case). + +### Notes + +This document addresses the special case of the configuration of the +`tools` package, mechanisms presented here could be extended to any +dependency that needs to be configured (e.g., detecting the installed +libncurse) but that is out of the scope of this document. + +Anywhere in this document we refer to the `tools` package as the +package that will receive the current `tools` package content, it does +not commit to keep that package name. + +## III. Requirements + +### `bazel init` should: + - _a1._ Not be available in hermetic version (i.e. Google version of + Bazel, a.k.a Blaze). + - _a2._ Allow per-language configuration. I.e., Java and C++ tooling + configuration should be separated. + - _a3._ Allow Skylark add-ons to specify their configuration, this + should be pluggable so we can actually activate configuration per + rule set. + - _a4._ Support at least 3 modes corresponding to each envisioned + use-cases: + * __UC1.__: __installed__ (default mode): a "hidden" `tools` + package contains the detected tool paths (`gcc`, the JDKs, ...) + as well as their configuration (basically the content of the + current `//tools` package). This package should be constructed + as much as possibly automatically with a way for the user to + overwrite detected settings. + * __UC2.__: __semi-hermetic__: the "hidden" `tools` package is used + only for linking the actual tool paths but the configuration + would be checked-in into the workspace (in a similar way that + what is currently done in Bazel). The "hidden" `tools` package + could contains several versions of the same tools (e.g., jdk-8, + jdk-7, ...) and the workspace link to a specific one. + * __UC3.__: __hermetic__: this is the Google way of thing: the + user check-in everything that he thinks belong to the workspace + and the init command should do nothing. + - _a5._ Support explicit reconfiguration. If the configuration + mechanism changes or the user wants to tune the configuration, it + should support to modify the configuration, i.e., update the + various paths or change the default options. + +### `bazel init` could: + + - _b1._: Initialize a new workspace: as it would support configuring + a whole tool directory, it might be quite close to actually + initializing a new workspace. + +## IV. User interface + +To be efficient, when the `tools` directory is missing, `bazel build` +should display an informative error message to actually run `bazel +init`. + +Configuration is basically just setting a list of build constants like +the path to the JDK, the list of C++ flags, etc... + +When the user type `bazel init`, the configuration process starts with +the default configuration (e.g., configure for “gold features” such as +C++, Java, sh_, ...). It should try to autodetect as much as possible. +If a language configuration needs something it cannot autodetect, then +it can prompt the user for the missing information and the +configuration can fail if something is really wrong. + +On default installation, `bazel init` should not prompt the user at +all. When the process finishes, the command should output a summary of +the configuration. The configuration is then stored in a "hidden" +directory which is similar to our current `tools` package. By default, +the labels in the configuration would direct to that package (always +mapped as a top-level package). The "hidden:" directory would live in +`$(output_base)/init/tools` and be mapped using the package path +mechanism. The `--overwrite` option would be needed to rerun the +automatic detection and overwrite everything including the eventual +user-set options. + +For the hermetic mode, the user has to recreate the default tools +package inside the workspace. If the user has a package with the same +name in the workspace, then the "hidden" directory should be ignored +(--package_path). + +To set a configuration option, the user would type `bazel init +java:jdk=/path/to/jdk` or to use the autodetection on a specific +option `bazel init java:jdk`. The list of settings group could be +obtained by `bazel init list` and the list of option with their value +for a specific language by `bazel init list group`. `bazel init list +all` could give the full configuration of all activated groups. + +_Prospective idea:_ Bazel init should explore the BUILD file to find the +Skylark `load` statements, determine if there is an associated init +script and use it. + +## V. Developer interface + +This section presents the support for developer that wants to add +autoconfiguration for a ruleset. The developer adding a configuration +would provide with a configuration script for it. This script will be +in charge of creating the package in the tools directory during `bazel +init` (i.e., the script for Java support will construct the +//tools/jdk package in the "hidden" package path). + +Because of skylark rules and the fact that the configuration script +should run before having access to the C++ and Java tooling, this +seems unreasonable to use a compiled language (Java or C++) for this +script. We could use the Skylark support to make it a subset of python +or we could use a bash script. Python support would be portable since +provided by Bazel itself and consistent with skylark. It also gives +immediate support for manipulating BUILD files. So keeping a +"skylark-like" syntax, the interface would look like: + +```python +configuration( + name, # name of the tools package to configure + autodetect_method, # the auto detection method + generate_method, # the actual package generation + load_method, # A method to load the attributes presented + # to the user from the package + attrs = { # List of attributes this script propose + "jdk_path": String, + "__some_other_path": String, # not user-settable + "jdk_version": Integer, + }) +``` + +Given that interface, an initial run of `bazel init` would do: + + 1. Find all language configuration scripts + 2. Run `load_method` for each script + 3. Run `autodetect_method` for each script. Replace non loaded + attribute (attribute still undefined after `load_method`) if and + only if `--rerun` option is provided + 4. Run `generate_method` for each script + 5. Fetch all non up to date dependencies of remote repository + +See Appendix B for examples of such methods. + +## VI. Implementation plan + + 1. Add the hidden tools directory and have it binded with package + path when no tools directory exists. The hidden tools directory + will have a WORKSPACE file and will have an automatic local + repository with the "init" name so that we can actually bind + targets from it into our workspace. + 2. Add `bazel init` that support the configuration for native + packages in Java, that is: Java, C++, genrule and test. This + would create the necessary mechanisms for supporting the + developer and the basic user interface. This commands will be + totally in Java for now and should trigger the fetch part of the + remote repository. + 3. Design and implement the language extension ala Skylark using the + design for the Java version of point 2. + 4. Convert the existing configuration into that language. + 5. Integrate the configuration with Skylark (i.e. Skylark rules + writer can add configuration step). We should here decide on + how it should be included (as a separate script? how do we + ship a skylark rule set? can we have load statement loading + a full set of rules?). + 6. Create configuration for the existing skylark build rules. If + we support load statement with label, we can then create a + repository for Skylark rules. + +## Appendix A. Various comments + + 1. We should get rid of the requirement for a `tools/defaults/BUILD` file. + 2. To works correctly, we need some local caching of the bazel + repository so tools are available. We could have bazelrc specify a + global path to the local cache (with `/etc/bazel.bazelrc` being loaded + first to `~/.bazelrc`). We could use a `~/.bazel` directory to put an + updatable tools cache also. This is needed because user probably want + to initialize a workspace tooling on a plane + 3. This proposal would probably add a new top-level package. We + should really take care of the naming convention for default top + packages (i.e., `tools`, `tools/defaults`, `visibility`, `external`, + `condition`...). We are going to make some user unhappy if they cannot + have an `external` directory at the top of their workspace (I would + just not use a build system that goes against my workspace structure). + While it is still time to do it, we should rename them with a nice + naming convention. A good way to do it is to make top-package name + constants, possibly settable in the WORKSPACE file (so we can actually + keep the name we like but user that are bothered by that can change + it). + 4. As we will remove the tools directory from the workspace, it + makes sense to add another prelude_bazel file somewhere else. As the + `WORKSPACE` file controls the workspace, it makes sense to have the + `prelude_bazel` logic in it (and the load statement should support + labels so that a user can actually specify remote repository labels). + +## Appendix B. Skylark-like code examples of configuration functions + +This is just a quick draft, please feel free to propose improvements: + +```python +# env is the environment, attrs are the values set either from the command-line +# or from loading the package +def autodetect_method(env, attrs): + if not attrs.java_version: # If not given in the command line nor loaded + attrs.java_version = 8 + if not attrs.jdk_path: + if env.has("JDK_HOME"): + attrs.jdk_path = env.get("JDK_HOME") + elif env.os = "darwin": + attrs.jdk_path = system("/usr/libexec/java_home -v 1." + attrs.java_version + "+") + else: + attrs.jdk_path = basename(basename(readlink(env.path.find(java)))) + if not attrs.jdk_path: + fail("Could not find JDK home, please set it with `bazel init java:jdk_path=/path/to/jdk`") + attrs.__some_other_path = first(glob(["/usr/bin/java", "/usr/local/bin/java"])) + + +# attrs is the list of attributes. It basically contains the list of rules +# we should generate in the corresponding package. Please note +# That all labels are replaced by relative ones as it should not be able +# to write out of the package. +def generate_method(attrs): + scratch_file("BUILD.jdk", """ +Content of the jdk BUILD file. +""") + # Create binding using local_repository. This should not lie in + # the WORKSPACE file but in a separate WORKSPACE file in the hidden + # directory. + local_repository(name = "jdk", path = attrs.jdk_path, build_file = "BUILD.jdk") + bind("@jdk//jdk", "jdk") # also add a filegroup("jdk", "//external:jdk") + java_toolchain(name = "toolchain", source = attrs.java_version, target = attrs.java_version) + # The magic __BAZEL_*__ variable could be set so we don’t + # redownload the repository if possible. This install_target + # should leverage the work already done on remote repositories. + # This should build and copy the result into the tools directory with + # The corresponding exports_files now. + install_target(__BAZEL_REPOSITORY__, __BAZEL_VERSION__, "//src/java_tools/buildjar:JavaBuilder_deploy.jar") + install_target(__BAZEL_REPOSITORY__, __BAZEL_VERSION__, "//src/java_tools/buildjar:JavaBuilder_deploy.jar") + copy("https://ijar_url", "ijar") + + + +# Load the package attributes. +# - attrs should be written and value will be replaced by the user-provided +# one if any +# - query is a query object restricted to the target package and resolving label +# relatively to the target package. This object should also be able to search +# for repository binding +# Note that the query will resolve in the actual tools directory, not the hidden +# one if it exists whereas the generation only happens in the hidden one. +def load_method(attrs, query): + java_toolchain = query.getOne(kind("java_toolchain", "...")) + if java_toolchain: + attrs.jdk_version = max(java_toolchain.source, java_toolchain.target) + jdk = query.getOne(attr("name", "jdk", kind("local_repository", "..."))) + if jdk: + attrs.jdk_path = jdk.path +``` diff --git a/site/designs/_posts/2015-07-02-skylark-remote-repositories.md b/site/designs/_posts/2015-07-02-skylark-remote-repositories.md new file mode 100644 index 0000000000..96e5fe0693 --- /dev/null +++ b/site/designs/_posts/2015-07-02-skylark-remote-repositories.md @@ -0,0 +1,161 @@ +--- +layout: contribute +title: Skylark Remote Repositories +--- + +# Skylark Remote Repositories + +__Status:__ implemented + +__Author:__ dmarting@google.com + +__Reviewers:__ lberki@google.com, laurentlb@google.com, kchodorow@google.com + +## Context + +[Skylark](/docs/skylark/index.html) is the +extension language for Bazel and lets Bazel users describe the +build for new languages easily. External users do not create +native rules and we want to avoid them doing so. + +[Remote repositories](/docs/external.html) +are a convenient way to specify your third party dependencies +and to fetch them along with the build if you don’t want to +check them in your repository. + +This document discuss “Skylark Remote Repositories”, that is +creating new remote repository rules using Skylark. + +## Why? + + - Enable users to specify new kind of repositories, we already have + requests for PyPI for example. We don’t want to be supporting every + kind of repository that exists outside. + - Enable Skylark rules to write macros to have one-liners for + including all their tools in your WORKSPACE file. + - Enable configuration of languages tooling: + [bazel init](/design/2015/03/06/bazel-init.html) + first approach as a separate tools is not really user-friendly and + the same kind of flexibility can be achieved by creating + repositories rule in Skylark. An example for the JDK is + [here](https://github.com/damienmg/bazel/commit/c40bd9c71965f0a8c85b732a0b0c3dbe31e017cf/jdk.bzl). + +## User interface (see the [JDK example](https://github.com/damienmg/bazel/commit/c40bd9c71965f0a8c85b732a0b0c3dbe31e017cf/jdk.bzl)) + +The load statement will now be available from the WORKSPACE, working +the same way it does for build file but with WORKSPACE specific +functions instead. + +In the same way that we have macros and rules for the BUILD file, we +are going to have macros and rule for the WORKSPACE file. The former +will be a convenient way to combine remote repositories and the latter +enable creation of new repositories kind. + +### Macros + +Skylark macros would be activated in the WORKSPACE file and would +behave as expected. Macros would enable to combine remote repositories +creation and bind into a single rules E.g. `setup_java()` would set-up +all bindings and the local repository needed to build java target: + +```python +def setup_java(): + native.new_local_repository(name = “jdk-local”, path = “/usr/share/java/jdk8”, build_file = “jdk.BUILD”) + for target in ["jni_header", "jni_md_header", "langtools", "bootclasspath", "extdir", "toolchain", "jdk", "java", "javac", "jar"]: + native.bind(name=target, actual="@%s//:%s" % (name, target)) + native.bind(name="jni_md_header-linux", actual="@%s//:jni_md_header" % name) + native.bind(name="jni_md_header-darwin", actual="@%s//:jni_md_header" % name) +``` + +### Remote repository rule +A remote repository rule would be set-up the same way we set-up a +build rule but with the `repository_rule` statement: + +```python +jdk_repository = repository_rule( + implementation = my_impl, + attrs = { + “java_home”: attr.string(mandatory=False), + “java_version”: attr.string(default=“1.8”), + } +``` + +This statement takes only 2 arguments: an implementation function and +a list of attributes. The syntax is similar to the rule statement but +attributes can only takes primitive type (String, Label, Integer, +Boolean, …) and not artifacts. + +The implementation function takes exactly one argument: the repository +context. This context will provides many convenience methods for doing +non hermetic operations, e.g., : + +- For discovering the environment: + * access system environment (`ctxt.os`), + * execute a program and get the standard output (`ctxt.execute`) to + discover the environment, + * … +- For creating the remote repository: + * fetch an artifact from URL (`ctxt.download`), + * uncompress an artifact (`ctxt.path(...).uncompress(outputPath)`), + * “copy” a directory from the system (`ctx.fetch_path(...)`), + * create a build file (`ctxt.build_file(...)`) + * ... + +The precise list of methods the repository context will support will +be augmented on-demand depending on what makes sense for our users. + +## How? + +A preliminary quick and dirty prototype can be found +[here](https://github.com/damienmg/bazel/commit/3a483ab53e547ae677599215faa9e3398bbd06ba) +and +[here](https://github.com/damienmg/bazel/commit/c40bd9c71965f0a8c85b732a0b0c3dbe31e017cf). + +Here what the prototype does: + + 1. [First commit](https://github.com/damienmg/bazel/commit/3a483ab53e547ae677599215faa9e3398bbd06ba) + activate Skylark macros and repositories + a. Allow Skylark load statements in the WORKSPACE file by adding + the various hook and a WorkspaceContext. + b. A new repository_rule in Skylark that can be called only from + the WORKSPACE file. + c. A new repository context that is passed to repository rule and + that should contain all the non-hermetic stuff so the rest of + skylark stays hermetic. + d. A bit of hack for tweaking the SkylarkNativeModule when in + WORKSPACE file to comply with the structure of the WORKSPACE + rules. + e. A dirty hack to load the SkylarkRepositoryFunction as a Skylark + module without breaking the package boundaries. This is due of + technical debts on loading Skylark module nicely (there is a TODO + to do it correctly). + 2. [Second commit](https://github.com/damienmg/bazel/commit/c40bd9c71965f0a8c85b732a0b0c3dbe31e017cf) + showcase the usage of Skylark remote repositories as a configuration step. + a. Add an example for fetching JDK dependencies. It does both the + detection and the fetching. + b. Add the necessary methods in the SkylarkRepositoryContext for + making the example work. + c. Added the search method to the Skylark string object (to do a + regex search). + +## Roadmap + +The obvious choice for the roadmap is to remake all those works, +correctly commented and tested, and then add methods to the +SkylarkRepositoryContext for full support. +More precisely the correct order of the work should be: + +1. Activate Skylark Macros taking part of 1.a and doing correctly 1.d + [[DONE](https://github.com/bazelbuild/bazel/commit/8ca065c2ae941309b53f95313a68e585ecc70560)] +2. Fix Skylark module load (1.e) + [[DONE](https://github.com/bazelbuild/bazel/commit/f81c675928c6beeaae5f66480dc7dbef47f75fb8)] +3. Add the SkylarkRepositoryFunction and empty context (1.b and 1.c) + [[DONE](https://github.com/bazelbuild/bazel/commit/653df8813dd74042e8e084eeae238a8b9f16a3ca)] +4. [Extends SkylarkRepositoryContext for handling C++ configuration](/designs/2016/02/16/cpp-autoconf.html) + [[DONE](https://github.com/bazelbuild/bazel/commit/8fa5ae6a6364100f2a7f9130e62eb0edb447339a)] +5. Extends SkylarkRepositoryContext + [[DONE](https://github.com/bazelbuild/bazel/commit/97f2c842ae706eccd822d8d264aba66c6102bade)] + for handling PyPI +6. Document + [[DONE](/docs/skylark/repository_rules.html)] +7. Extends SkylarkRepositoryContext for handling Docker pull diff --git a/site/designs/_posts/2016-02-16-cpp-autoconf.md b/site/designs/_posts/2016-02-16-cpp-autoconf.md new file mode 100644 index 0000000000..c80fc8cb62 --- /dev/null +++ b/site/designs/_posts/2016-02-16-cpp-autoconf.md @@ -0,0 +1,145 @@ +--- +layout: contribute +title: Generating C++ crosstool with a Skylark Remote Repository +--- + +# Generating C++ crosstool with a Skylark Remote Repository + +__Status:__ implemented + +__Author:__ dmarting@google.com + +__Reviewers:__ lberki@google.com + +## Context + +[Skylark](/docs/skylark/index.html) is the +extension language for Bazel and lets Bazel users describe the +build for new languages easily. External users do not create +native rules and we want to avoid them doing so. + +[Remote repositories](/docs/external.html) +are a convenient way to specify your third party dependencies +and to fetch them along with the build if you don’t want to +check them in your repository. + +[Skylark remote +repositories](/designs/2015/07/02/skylark-remote-repositories.html) is +an ongoing effort to support specifying new remote repositories using +Skylark. + +## Why? + +Configurability issues are stopping users from compiling and using +Bazel on complex setup. In particular, +[TensorFlow](https://tensorflow.io)’s users runs on +various hardware where gcc is installed on non-standard directory that +needs to change the +[CROSSTOOL](https://github.com/bazelbuild/bazel/blob/6b6ff76945c80fb8b11b71d402b5146c85b86859/tools/cpp/CROSSTOOL) +file (see +[issue #531](https://github.com/bazelbuild/bazel/issues/531)). +This generally requires to change the list of include directories, the +path to gcc and sometimes also the linking option to find the correct +libraries at runtime. Some platform even requires +[special wrappers around gcc](https://github.com/bazelbuild/bazel/blob/6b6ff76945c80fb8b11b71d402b5146c85b86859/tools/cpp/osx_gcc_wrapper.sh). + +Java solved the problem by setting a custom repository +([@local_jdk](https://github.com/bazelbuild/bazel/blob/6b6ff76945c80fb8b11b71d402b5146c85b86859/src/main/java/com/google/devtools/build/lib/bazel/rules/java/jdk.WORKSPACE#L3)) +where the path is automatically detected using the [location of the JVM +running +Bazel](https://github.com/bazelbuild/bazel/blob/6b6ff76945c80fb8b11b71d402b5146c85b86859/src/main/java/com/google/devtools/build/lib/packages/WorkspaceFactory.java#L414). +But this approach does not scale well with complex language setup like +C++. + +We reduced the number of C++ issues the user had with removing all C++ +compilation in the bootstrap of Bazel. However, to properly handle +those platform, Bazel needs some level of auto-configuration +([Kythe](https://github.com/google/kythe/blob/a29f0adc6fa11550f66bc2278f17b89b9e02de18/setup_bazel.sh) +and +[Tensorflow](https://github.com/tensorflow/tensorflow/blob/a81c4f9cd01563e97fc6f179e4d70960fc9b02ae/configure) +have their own auto-configuration scripts). This document discuss how +to use a skylark remote repository to implement a simple +auto-configuration for C++ crosstool (step 4 of the roadmap from the +[Skylark remote +repositories](/design/2015/07/02/skylark-remote-repositories.html) +document). + +## C++ toolchain detection + +Until now here the various issues user have faced using a custom C++ +toolchain: + + 1. C++ compiler is not at the expected location. + 2. C++ compiler is `clang` and not `gcc` or behaves differently than + what Bazel C++ rules expect. + 3. Libraries are not in the default location. + 4. Headers are not in the default location. + 5. Path of libraries or headers are outside of the default mounted + paths. + +The current fix we propose to the user for the various issue are: + + 1. Change the tool paths in + [tools/cpp/CROSSTOOL#L87](https://github.com/bazelbuild/bazel/blob/6b6ff76945c80fb8b11b71d402b5146c85b86859/tools/cpp/CROSSTOOL#L87). + 2. Add a wrapper like + [tools/cpp/osx\_gcc\_wrapper.sh](https://github.com/bazelbuild/bazel/blob/6b6ff76945c80fb8b11b71d402b5146c85b86859/tools/cpp/osx_gcc_wrapper.sh) + and modify some options from the CROSSTOOL file. + 3. Add `-Wl,rpath,` option to the + [linker\_flags](https://github.com/bazelbuild/bazel/blob/6b6ff76945c80fb8b11b71d402b5146c85b86859/tools/cpp/CROSSTOOL#L93). + 4. Add the directories to the + [cxx\_builtin\_include\_dirs](https://github.com/bazelbuild/bazel/blob/6b6ff76945c80fb8b11b71d402b5146c85b86859/tools/cpp/CROSSTOOL#L100). + 5. Deactivate sandboxing + +To address those issues, we propose to add the followings methods to +the repository context object: + + 1. `which(cmd)` returns the path to the binary designed by `cmd`, + looking for it in the path environment variable (or equivalent). + This will help getting the path to the C++ compiler. + 2. `execute([arg0, arg1, ..., argn])` executes a command and returns an + `exec_result` struct containing: + * `stdout` the content of the standard output, + * `stderr` the content of the standard error output, and + * `return_code` the return code of the execution. + Executing `$(CC)` will help detect whether we are using gcc or + clang. + 3. An `os` object with an environ map containing the list of + environment variable. The os object will be extended to + contains all OS specific variables (platform name and much more). + 4. `execute([..])` from 2 will be used to run [`gcc ... +-v`](http://stackoverflow.com/questions/11946294/dump-include-paths-from-g) + to list the built-in include directories. +To address the issue 5, we can add the list of paths to dependencies to the +[crosstool rule in the BUILD +file](https://github.com/bazelbuild/bazel/wiki/Building-with-a-custom-toolchain). + +## Writing the cpp package + +Once we have resolved all the information from the system, we need to +write two or three files: + + - The `BUILD` file that will contains the corresponding + `cc_toolchain` rules + - The `CROSSTOOL` file + - Optionally, the wrapper script. + +We should extends the context with a `file(path, content)` method, where +path is a path relative to the repository root and content the content +of the file to write. + +To ease the writing of crosstool, we should also provide a +`template(path, label, variables)` method which will write the file +pointed by path using the file pointed by label (should be a +FileValue) in which variables have been replaced in the same way that +[template_action](http://bazel.io/docs/skylark/lib/ctx.html#template_action) +works. + +## Rollout plan + +The implementation plan would be: + + 1. Implements `which`, `execute`, `os`, `file` and `template` + [__DONE__] + 2. Write the `cc_configure` repository rule which does the work. Use + GitHub bugs as inputs on which platform to support. [__DONE__] + 3. Advertise the existence of `cc_configure` [__DONE__] |