// Copyright 2016 The Bazel Authors. All rights reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package com.google.devtools.build.lib.runtime.commands; import com.google.common.collect.ImmutableList; import com.google.devtools.build.lib.packages.TestTimeout; import com.google.devtools.build.lib.runtime.Command; import com.google.devtools.common.options.OptionPriority; import com.google.devtools.common.options.OptionsParser; import com.google.devtools.common.options.OptionsParsingException; /** * Handles the 'coverage' command on the Bazel command line. * *
Here follows a brief, partial and probably wrong description of how coverage collection works * in Bazel. * *
Coverage is reported by the tests in LCOV format in the files * {@code testlogs/PACKAGE/TARGET/coverage.dat} and * {@code testlogs/PACKAGE/TARGET/coverage.micro.dat}. * *
To collect coverage, each test execution is wrapped in a script called * {@code collect_coverage.sh}. This script sets up the environment of the test to enable coverage * collection and determine where the coverage files are written by the coverage runtime(s). It * then runs the test. A test may itself run multiple subprocesses and consist of modules written * in multiple different languages (with separate coverage runtimes). As such, the wrapper script * converts the resulting files to lcov format if necessary, and merges them into a single file. * *
The interposition itself is done by the test strategies, which requires * {@code collect_coverage.sh} to be on the inputs of the test. This is accomplished by an implicit * attribute {@code :coverage_support} which is resolved to the value of the configuration flag * {@code --coverage_support} (see {@link * com.google.devtools.build.lib.analysis.config.BuildConfiguration.Options#coverageSupport}). * *
There are languages for which we do offline instrumentation, meaning that the coverage * instrumentation is added at compile time, e.g. for C++, and for others, we do online * instrumentation, meaning that coverage instrumentation is added at execution time, e.g. for * Javascript. * *
Another core concept is that of baseline coverage. This is essentially the coverage of * library, binary, or test if no code in it was run. The problem it solves is that if you want to * compute the test coverage for a binary, it is not enough to merge the coverage of all of the * tests, because there may be code in the binary that is not linked into any test. Therefore, what * we do is to emit a coverage file for every binary, which contains only the files we collect * coverage for with no covered lines. The baseline coverage file for a target is at * {@code testlogs/PACKAGE/TARGET/baseline_coverage.dat}. Note that it is also generated for * binaries and libraries in addition to tests if you pass the {@code --nobuild_tests_only} flag to * Bazel. * *
Baseline coverage collection is currently broken. * *
We track two groups of files for coverage collection for each rule: the set of instrumented * files and the set of instrumentation metadata files. * *
The set of instrumented files is just that, a set of files to instrument. For online coverage * runtimes, this can be used at runtime to decide which files to instrument. It is also used to * implement baseline coverage. * *
The set of instrumentation metadata files is the set of extra files a test needs to generate * the LCOV files Bazel requires from it. In practice, this consists of runtime-specific files; for * example, the gcc compiler emits {@code .gcno} files during compilation. These are added to the * set of inputs of test actions if coverage mode is enabled (otherwise the set of metadata files * is empty). * *
Whether or not coverage is being collected is stored in the {@code BuildConfiguration}. This * is handy because then we have an easy way to change the test action and the action graph * depending on this bit, but it also means that if this bit is flipped, all targets need to be * re-analyzed (note that some languages, e.g. C++ require different compiler options to emit * code that can collect coverage, which dominates the time required for analysis). * *
The coverage support files are depended on through labels in {@code //tools/defaults} and set * through command-line options, so that they can be overridden by the invocation policy, which * allows them to differ between the different versions of Bazel. Ideally, these differences will * be removed, and we standardize on @bazel_tools//tools/coverage. * *
A partial set of file types that can be encountered in the coverage world: *
OPEN QUESTIONS: *