Merged commit includes the following changes:

175703479  by yifeif:

    Internal change.

--
175695370  by A. Unique TensorFlower:

    Implements _shared_embedding_columns and adds some tests.

--
175695349  by A. Unique TensorFlower:

    Implements tf.metrics.true_negatives, adds missing tests and does some cleanup in tf.contrib.metrics.

--

PiperOrigin-RevId: 175703479

1 files changed, 417 insertions, 0 deletions

diff --git a/tensorflow/contrib/lite/g3doc/tf_ops_compatibility.md b/tensorflow/contrib/lite/g3doc/tf_ops_compatibility.md
new file mode 100644
index 0000000000..121c4c2c95
--- /dev/null
+++ b/tensorflow/contrib/lite/g3doc/tf_ops_compatibility.md
@@ -0,0 +1,417 @@
+# TensorFlow Compatibility Guide
+
+TensorFlow Lite supports a number of TensorFlow operations used in common
+inference models. As they are processed by the TensorFlow Lite Optimizing
+Converter, those operations may be elided or fused, before the supported
+operations are mapped to their TensorFlow Lite counterparts.
+
+Since the set of TensorFlow Lite operations is smaller than TensorFlow's, not
+every model is convertible. Even for supported operations, very specific usage
+patterns are sometimes expected, for performance reasons. We expect to expand
+the set of supported operations in future TensorFlow Lite releases.
+
+The best way to understand how to build a TensorFlow model that can be used with
+TensorFlow Lite is to carefully consider how operations are converted and
+optimized, along with the limitations imposed by this process.
+
+## Supported Types
+
+Most TensorFlow Lite operations target both floating-point (float32) and
+quantized (uint8) inference, but usually there is little or no support for other
+types like tf.float16 and strings.
+
+Apart from using different version of the operations, the other difference
+between floating-point and quantized models lies in the way they are converted.
+Quantized conversion expect the models to be annotated with "fake quantization"
+nodes that record the dynamic range of the tensors. Without that information TF
+Lite is not able to accurately quantize a model, which means that proper
+quantized training is necessary before conversion.
+
+## Data Format and Broadcasting
+
+At the moment TensorFlow Lite supports only TensorFlow's "NHWC" format, and
+broadcasting in operations like tf.add and tf.mul is generally not supported.
+
+## Compatible Operations
+
+The following TensorFlow operations are usually mapped to their TensorFlow Lite
+counterparts:
+
+*   [tf.matmul](https://www.tensorflow.org/api_docs/python/tf/matmul) - *as long
+    as the second argument is constant and transposition is not used*
+*   [tf.nn.avg_pool](https://www.tensorflow.org/api_docs/python/tf/nn/avg_pool)
+*   [tf.nn.conv2d](https://www.tensorflow.org/api_docs/python/tf/nn/conv2d) -
+    *as long as the filter is constant*
+*   [tf.nn.depthwise_conv2d](https://www.tensorflow.org/api_docs/python/tf/nn/depthwise_conv2d) -
+    *as long as the filter is constant and rate is [1,1]*
+*   [tf.nn.l2_normalize](https://www.tensorflow.org/api_docs/python/tf/nn/l2_normalize) -
+    *as long as normalization is done along the last dimension*
+*   [tf.nn.local_response_normalization](https://www.tensorflow.org/api_docs/python/tf/nn/local_response_normalization)
+*   [tf.nn.max_pool](https://www.tensorflow.org/api_docs/python/tf/nn/max_pool)
+*   [tf.nn.softmax](https://www.tensorflow.org/api_docs/python/tf/nn/softmax) -
+    *as long as tensors are 2D and axis is the last dimension*
+*   [tf.reshape](https://www.tensorflow.org/api_docs/python/tf/reshape)
+*   [tf.sigmoid](https://www.tensorflow.org/api_docs/python/tf/sigmoid)
+*   [tf.space_to_depth](https://www.tensorflow.org/api_docs/python/tf/space_to_depth)
+
+## Straighforward Conversions, Constant-Folding and Fusing
+
+A number of TensorFlow operations can be processed by TensorFlow Lite even
+though they have no direct equivalent. This is the case for operations that can
+be simply removed from the graph (tf.identity), replaced by tensors
+(tf.placeholder), or fused into more complex operations (tf.nn.bias_add). Even
+some supported operations may sometimes be removed through one of these
+processes.
+
+Here is a list of TensorFlow operations that are usually removed from the graph:
+
+*   [tf.add](https://www.tensorflow.org/api_docs/python/tf/add)
+*   [tf.check_numerics](https://www.tensorflow.org/api_docs/python/tf/check_numerics)
+*   [tf.constant](https://www.tensorflow.org/api_docs/python/tf/constant)
+*   [tf.div](https://www.tensorflow.org/api_docs/python/tf/div)
+*   [tf.divide](https://www.tensorflow.org/api_docs/python/tf/divide)
+*   [tf.fake_quant_with_min_max_args](https://www.tensorflow.org/api_docs/python/tf/fake_quant_with_min_max_args)
+*   [tf.fake_quant_with_min_max_vars](https://www.tensorflow.org/api_docs/python/tf/fake_quant_with_min_max_vars)
+*   [tf.greater](https://www.tensorflow.org/api_docs/python/tf/greater)
+*   [tf.greater_equal](https://www.tensorflow.org/api_docs/python/tf/greater_equal)
+*   [tf.identity](https://www.tensorflow.org/api_docs/python/tf/identity)
+*   [tf.less](https://www.tensorflow.org/api_docs/python/tf/less)
+*   [tf.less_equal](https://www.tensorflow.org/api_docs/python/tf/less_equal)
+*   [tf.maximum](https://www.tensorflow.org/api_docs/python/tf/maximum)
+*   [tf.minimum](https://www.tensorflow.org/api_docs/python/tf/minimum)
+*   [tf.multiply](https://www.tensorflow.org/api_docs/python/tf/multiply)
+*   [tf.no_op](https://www.tensorflow.org/api_docs/python/tf/no_op)
+*   [tf.placeholder](https://www.tensorflow.org/api_docs/python/tf/placeholder)
+*   [tf.placeholder_with_default](https://www.tensorflow.org/api_docs/python/tf/placeholder_with_default)
+*   [tf.realdiv](https://www.tensorflow.org/api_docs/python/tf/realdiv)
+*   [tf.reduce_max](https://www.tensorflow.org/api_docs/python/tf/reduce_max)
+*   [tf.reduce_min](https://www.tensorflow.org/api_docs/python/tf/reduce_min)
+*   [tf.reduce_sum](https://www.tensorflow.org/api_docs/python/tf/reduce_sum)
+*   [tf.rsqrt](https://www.tensorflow.org/api_docs/python/tf/rsqrt)
+*   [tf.shape](https://www.tensorflow.org/api_docs/python/tf/shape)
+*   [tf.sqrt](https://www.tensorflow.org/api_docs/python/tf/sqrt)
+*   [tf.square](https://www.tensorflow.org/api_docs/python/tf/square)
+*   [tf.squeeze](https://www.tensorflow.org/api_docs/python/tf/squeeze)
+*   [tf.subtract](https://www.tensorflow.org/api_docs/python/tf/subtract)
+*   [tf.tile](https://www.tensorflow.org/api_docs/python/tf/tile)
+*   [tf.nn.batch_norm_with_global_normalization](https://www.tensorflow.org/api_docs/python/tf/nn/batch_norm_with_global_normalization)
+*   [tf.nn.bias_add](https://www.tensorflow.org/api_docs/python/tf/nn/bias_add)
+*   [tf.nn.fused_batch_norm](https://www.tensorflow.org/api_docs/python/tf/nn/fused_batch_norm)
+*   [tf.nn.relu](https://www.tensorflow.org/api_docs/python/tf/nn/relu)
+*   [tf.nn.relu6](https://www.tensorflow.org/api_docs/python/tf/nn/relu6)
+
+Note that many of those operations don't have TensorFlow Lite equivalents and
+the corresponding model will not be convertible if they can't be elided or
+fused.
+
+## Unsupported Operations
+
+TensorFlow operation not listed above are likely unsupported. Notably, the
+following common ops are not supported at the moment:
+
+*   [tf.batch_to_space_nd](https://www.tensorflow.org/api_docs/python/tf/batch_to_space_nd)
+*   [tf.depth_to_space](https://www.tensorflow.org/api_docs/python/tf/depth_to_space)
+*   [tf.floor](https://www.tensorflow.org/api_docs/python/tf/floor)
+*   [tf.gather](https://www.tensorflow.org/api_docs/python/tf/gather)
+*   [tf.image.resize_bilinear](https://www.tensorflow.org/api_docs/python/tf/image/resize_bilinear)
+*   [tf.pad](https://www.tensorflow.org/api_docs/python/tf/pad)
+*   [tf.reduce_mean](https://www.tensorflow.org/api_docs/python/tf/reduce_mean)
+*   [tf.slice](https://www.tensorflow.org/api_docs/python/tf/slice)
+*   [tf.space_to_batch_nd](https://www.tensorflow.org/api_docs/python/tf/space_to_batch_nd)
+*   [tf.split](https://www.tensorflow.org/api_docs/python/tf/split)
+*   [tf.strided_slice](https://www.tensorflow.org/api_docs/python/tf/strided_slice)
+*   [tf.tanh](https://www.tensorflow.org/api_docs/python/tf/tanh)
+
+## TensorFlow Lite Operations
+
+The following TensorFlow Lite operations are fully supported and used in place
+of the TensorFlow operations listed above:
+
+**ADD**
+
+```
+Inputs {
+  0: a tensor
+  1: a tensor
+}
+Outputs {
+  0: elementwise sum of the input tensors
+}
+Options {
+  fused_activation_function:  NONE|RELU|RELU6
+}
+```
+
+**AVERAGE_POOL_2D**
+
+```
+Inputs {
+  0: a tensor
+}
+Outputs {
+  0: a tensor where each entry is the mean of the input values in the
+     corresponding window.
+}
+Options {
+  fused_activation_function:  NONE|RELU|RELU6
+  padding: SAME|VALID
+  stride_w,stride_h: stride of the sliding window
+  filter_width,filter_height: size of the sliding window
+}
+```
+
+**CONCATENATION**
+
+```
+Inputs {
+  0-N: any number of tensors
+}
+Outputs {
+  0: concatenation of the input tensors along the given axis.
+}
+Options {
+  fused_activation_function:  NONE|RELU|RELU6
+  axis: dimension along which the concatenation is performed
+}
+```
+
+**CONV_2D**
+
+```
+Inputs {
+  0: 4D tensor
+  1: filter
+  2: bias (optional)
+}
+Outputs {
+  0: result of 2D convolution of the input tensor
+}
+Options {
+  fused_activation_function:  NONE|RELU|RELU6
+  padding: SAME|VALID
+  stride_w,stride_h: stride of the filter window
+}
+```
+
+**DEPTHWISE_CONV_2D**
+
+```
+Inputs {
+  0: 4D tensor
+  1: filter
+  2: bias (optional)
+}
+Outputs {
+  0: result of a depthwise-2D convolution of the input tensor
+}
+Options {
+  fused_activation_function:  NONE|RELU|RELU6
+  padding: SAME|VALID
+  stride_w,stride_h: stride of the filter window
+  depth_multiplier: relation between the last dimension of the input and output
+    tensors
+}
+```
+
+**FULLY_CONNECTED**
+
+```
+Inputs {
+  0: 4D tensor
+  1: filter
+  2: bias (optional)
+}
+Outputs {
+  0: output of a fully (densely) connected layer, which connects all
+     elements in the input tensor with each element in this tensor.
+}
+Options {
+  fused_activation_function:  NONE|RELU|RELU6
+}
+```
+
+**L2_NORMALIZATION**
+
+```
+Inputs {
+  0: input tensor
+}
+Outputs {
+  0: normalized tensor (along the last dimension)
+}
+Options {
+  fused_activation_function:  NONE|RELU|RELU6
+}
+```
+
+**L2_POOL_2D**
+
+```
+Inputs {
+  0: a tensor
+}
+Outputs {
+  0: a tensor equivalent to tf.sqrt(tf.nn.ave_pool(tf.square(input))
+}
+Options {
+  fused_activation_function:  NONE|RELU|RELU6
+  padding: SAME|VALID
+  stride_w,stride_h: stride of the sliding window
+  filter_width,filter_height: size of the sliding window
+}
+```
+
+**LOCAL_RESPONSE_NORMALIZATION**
+
+```
+Inputs {
+  0: a tensor
+}
+Outputs {
+  0: a tensor equivalent to tf.nn.local_response_normalization
+}
+Options {
+  radius
+  bias
+  alpha
+  beta
+}
+```
+
+**LOGISTIC**
+
+```
+Inputs {
+  0: a tensor
+}
+Outputs {
+  0: a tensor equivalent to 1 / (1 + exp(-input))
+}
+```
+
+**MAX_POOL_2D**
+
+```
+Inputs {
+  0: a tensor
+}
+Outputs {
+  0: a tensor where each entry is the maximum of the input values in the
+     corresponding window.
+}
+Options {
+  fused_activation_function:  NONE|RELU|RELU6
+  padding: SAME|VALID
+  stride_w,stride_h: stride of the sliding window
+  filter_width,filter_height: size of the sliding window
+}
+```
+
+**MUL**
+
+```
+Inputs {
+  0: a tensor
+  1: a tensor
+}
+Outputs {
+  0: elementwise multiplication of the input tensors
+}
+Options {
+  fused_activation_function:  NONE|RELU|RELU6
+}
+```
+
+**RELU**
+
+```
+Inputs {
+  0: a tensor
+}
+Outputs {
+  0: a tensor equivalent to max(0, min(input, 1)
+}
+```
+
+**RELU1**
+
+```
+Inputs {
+  0: a tensor
+}
+Outputs {
+  0: a tensor equivalent to max(-1, min(input, 6)
+}
+```
+
+**RELU6**
+
+```
+Inputs {
+  0: a tensor
+}
+Outputs {
+  0: a tensor equivalent to max(0, min(input, 6)
+}
+```
+
+**RESHAPE**
+
+```
+Inputs {
+  0: a tensor
+  1: ignored
+}
+Outputs {
+  0: a tensor with the same elements as the input but with the new shape
+}
+Options {
+  new_shape
+}
+```
+
+**SOFTMAX**
+
+```
+Inputs {
+  0: a tensor
+}
+Outputs {
+  0: a tensor equivalent to exp(input) / tf.reduce_sum(exp(input * beta), dim),
+     where dim is always the last dimension of the input tensor.
+}
+Options {
+  beta
+}
+```
+
+**SPACE_TO_DEPTH**
+
+```
+Inputs {
+  0: a 4D tensor
+}
+Outputs {
+  0: a tensor rearranged using block_size. See tf.space_to_depth for details.
+}
+Options {
+  block_size
+}
+```
+
+And these are TensorFlow Lite operations that are present but not ready for
+custom models yet:
+
+*   CALL
+*   CONCAT_EMBEDDINGS
+*   CUSTOM
+*   EMBEDDING_LOOKUP
+*   EMBEDDING_LOOKUP_SPARSE
+*   HASHTABLE_LOOKUP
+*   LSH_PROJECTION
+*   LSTM
+*   RESIZE_BILINEAR
+*   RNN
+*   SKIP_GRAM
+*   SVDF
+*   TANH