Clarify Mutability invariants, refactor some tests

RELNOTES: None
PiperOrigin-RevId: 170343759

2 files changed, 66 insertions, 27 deletions

diff --git a/src/main/java/com/google/devtools/build/lib/syntax/Environment.java b/src/main/java/com/google/devtools/build/lib/syntax/Environment.java
index 2c0a2a6f4e..8abf9428fd 100644
--- a/src/main/java/com/google/devtools/build/lib/syntax/Environment.java
+++ b/src/main/java/com/google/devtools/build/lib/syntax/Environment.java
@@ -40,24 +40,25 @@ import java.util.TreeSet;
 import javax.annotation.Nullable;
 
 /**
- * An Environment is the main entry point to evaluating code in the BUILD language or Skylark. It
- * embodies all the state that is required to evaluate such code, except for the current instruction
- * pointer, which is an {@link ASTNode} whose {@link Statement#exec exec} or {@link Expression#eval
- * eval} method is invoked with this Environment, in a straightforward direct-style AST-walking
- * interpreter. {@link Continuation}-s are explicitly represented, but only partly, with another
- * part being implicit in a series of try-catch statements, to maintain the direct style. One
- * notable trick is how a {@link UserDefinedFunction} implements returning values as the function
- * catching a {@link ReturnStatement.ReturnException} thrown by a {@link ReturnStatement} in the
- * body.
+ * An {@code Environment} is the main entry point to evaluating Skylark code. It embodies all the
+ * state that is required to evaluate such code, except for the current instruction pointer, which
+ * is an {@link ASTNode} that is evaluated (for expressions) or executed (for statements) with
+ * respect to this {@code Environment}.
  *
- * <p>Every Environment has a {@link Mutability} field, and must be used within a function that
- * creates and closes this {@link Mutability} with the try-with-resource pattern. This {@link
- * Mutability} is also used when initializing mutable objects within that Environment; when closed
- * at the end of the computation freezes the Environment and all those objects that then become
- * forever immutable. The pattern enforces the discipline that there should be no dangling mutable
- * Environment, or concurrency between interacting Environment-s. It is also an error to try to
- * mutate an Environment and its objects from another Environment, before the {@link Mutability} is
- * closed.
+ * <p>{@link Continuation}-s are explicitly represented, but only partly, with another part being
+ * implicit in a series of try-catch statements, to maintain the direct style. One notable trick is
+ * how a {@link UserDefinedFunction} implements returning values as the function catching a {@link
+ * ReturnStatement.ReturnException} thrown by a {@link ReturnStatement} in the body.
+ *
+ * <p>Every {@code Environment} has a {@link Mutability} field, and must be used within a function
+ * that creates and closes this {@link Mutability} with the try-with-resource pattern. This {@link
+ * Mutability} is also used when initializing mutable objects within that {@code Environment}. When
+ * the {@code Mutability} is closed at the end of the computation, it freezes the {@code
+ * Environment} along with all of those objects. This pattern enforces the discipline that there
+ * should be no dangling mutable {@code Environment}, or concurrency between interacting {@code
+ * Environment}s. It is a Skylark-level error to attempt to mutate a frozen {@code Environment} or
+ * its objects, but it is a Java-level error to attempt to mutate an unfrozen {@code Environment} or
+ * its objects from within a different {@code Environment}.
  *
  * <p>One creates an Environment using the {@link #builder} function, then populates it with {@link
  * #setup}, {@link #setupDynamic} and sometimes {@link #setupOverride}, before to evaluate code in
diff --git a/src/main/java/com/google/devtools/build/lib/syntax/Mutability.java b/src/main/java/com/google/devtools/build/lib/syntax/Mutability.java
index 15fbe6f7ce..804fd48e57 100644
--- a/src/main/java/com/google/devtools/build/lib/syntax/Mutability.java
+++ b/src/main/java/com/google/devtools/build/lib/syntax/Mutability.java
@@ -31,13 +31,19 @@ import java.util.List;
  * {@code Mutability} instance. Once the {@code Environment} is done evaluating, its {@code
  * Mutability} is irreversibly closed ("frozen"). At that point, it is no longer possible to change
  * either the bindings in that {@code Environment} or the state of its objects. This protects each
- * {@code Environment} from unintentional and unsafe modification. Before freezing, only a single
- * thread may use the contents of the {@code Environment}, but after freezing, any number of threads
- * may access it.
+ * {@code Environment} from unintentional and unsafe modification.
  *
- * <p>It is illegal for an evaluation in one {@code Environment} to affect another {@code
- * Environment}, even if the second {@code Environment} has not yet been frozen. In practice, the
- * only unfrozen values that any {@code Environment} should be able to access are its own.
+ * <p>{@code Mutability}s enforce isolation between {@code Environment}s; it is illegal for an
+ * evaluation in one {@code Environment} to affect the bindings or values of another. In particular,
+ * the {@code Environment} for any Skylark module is frozen before its symbols can be imported for
+ * use by another module. Each individual {@code Environment}'s evaluation is single-threaded, so
+ * this isolation also translates to thread safety. Any number of threads may simultaneously access
+ * frozen data.
+ *
+ * <p>Although the mutability pointer of a {@code Freezable} contains some debugging information
+ * about its context, this should not affect the {@code Freezable}'s semantics. From a behavioral
+ * point of view, the only thing that matters is whether the {@code Mutability} is frozen, not what
+ * particular {@code Mutability} object is pointed to.
  *
  * <p>A {@code Mutability} also tracks which {@code Freezable} objects in its {@code Environment}
  * are temporarily locked from mutation. This is used to prevent modification of iterables during
@@ -45,7 +51,7 @@ import java.util.List;
  * iterable). Locking an object does not prohibit mutating its deeply contained values, such as in
  * the case of a list of lists.
  *
- * We follow two disciplines to ensure safety. First, all mutation methods of a {@code Freezable}
+ * <p>We follow two disciplines to ensure safety. First, all mutation methods of a {@code Freezable}
  * must take in a {@code Mutability} as a parameter, and confirm that
  * <ol>
  *   <li>the {@code Freezable} is not yet frozen,
@@ -67,9 +73,23 @@ import java.util.List;
  * block, relying on the try-with-resource construct to ensure that everything gets frozen before
  * the result is used. The only code that should create a {@code Mutability} without using
  * try-with-resource is test code that is not part of the Bazel jar.
+ *
+ * We keep some (unchecked) invariants regarding where {@code Mutability} objects may appear in a
+ * compound value.
+ * <ol>
+ *   <li>There is always at most one unfrozen {@code Mutability}, corresponding to the current
+ *       {@code Environment}'s evaluation.
+ *   <li>Whenever a new mutable Skylark value is created, its {@code Mutability} is either the
+ *       current {@code Environment}'s {@code Mutability}, or else it is the special static
+ *       instance, {@link #IMMUTABLE}, which represents that a value is at least shallowly
+ *       immutable.
+ * </ol>
+ * It follows that an unfrozen value can never appear as the child of a frozen value unless the
+ * frozen value's {@code Mutability} is {@code IMMUTABLE}. This can be used to prune traversals that
+ * check whether a value is deeply immutable.
  */
-// TODO(bazel-team): This safe usage pattern can be enforced through the use of a higher-order
-// function.
+// TODO(bazel-team): The safe try-with-resources usage pattern can be enforced through the use of a
+// higher-order function.
 public final class Mutability implements AutoCloseable, Serializable {
 
   /**
@@ -280,6 +300,24 @@ public final class Mutability implements AutoCloseable, Serializable {
     }
   }
 
-  /** A singular instance for permanently immutable things. */
+  /**
+   * An instance indicating that a value is shallowly immutable. Its children may or may not be
+   * mutable.
+   *
+   * <p>This instance is treated specially with regard to the {@code Mutability} invariant. Usually
+   * an immutable value cannot directly or indirectly contain a mutable one. But an immutable value
+   * with this {@code Mutability} may.
+   *
+   * <p>In practice, this instance is used as the {@code Mutability} for tuples. It may also be used
+   * for certain lists and dictionaries that are immutable from creation -- though in general we
+   * prefer to use tuples rather than always-frozen lists.
+   */
+  // TODO(bazel-team): We might be able to remove this instance, and instead have tuples and other
+  // always-immutable things store the same Mutability as other values in that environment. Then we
+  // can simplify the Mutability invariant, and implement deep-immutability checking in constant
+  // time.
+  //
+  // This would also affect structs (SkylarkInfo). Maybe they would implement an interface similar
+  // to SkylarkMutable, or the relevant methods could be worked into SkylarkValue.
   public static final Mutability IMMUTABLE = create("IMMUTABLE").freeze();
 }