diff options
| author |  Mark Daoust <markdaoust@google.com> | 2018-08-27 11:57:42 -0700 |
|---|---|---|
| committer |  TensorFlower Gardener <gardener@tensorflow.org> | 2018-08-27 12:01:50 -0700 |
| commit | 6c2bf6576321ad53ff1eb6d66b6efae2c93ef4e0 (patch) | |
| tree | eda82ca98d1ac04718c14c086e6221bd8d787b30 /tensorflow/docs_src/api_guides/python | |
| parent | abc8452394aeeecc1f3fef27f7098a5924bdd0e9 (diff) | |
Moved tensorflow/docs_src to https://github.com/tensorflow/docs
PiperOrigin-RevId: 210405729
Diffstat (limited to 'tensorflow/docs_src/api_guides/python')
43 files changed, 0 insertions, 4423 deletions
diff --git a/tensorflow/docs_src/api_guides/python/array_ops.md b/tensorflow/docs_src/api_guides/python/array_ops.md deleted file mode 100644 index ddeea80c56..0000000000 --- a/tensorflow/docs_src/api_guides/python/array_ops.md +++ /dev/null @@ -1,87 +0,0 @@ -# Tensor Transformations - -Note: Functions taking `Tensor` arguments can also take anything accepted by -`tf.convert_to_tensor`. - -[TOC] - -## Casting - -TensorFlow provides several operations that you can use to cast tensor data -types in your graph. - -* `tf.string_to_number` -* `tf.to_double` -* `tf.to_float` -* `tf.to_bfloat16` -* `tf.to_int32` -* `tf.to_int64` -* `tf.cast` -* `tf.bitcast` -* `tf.saturate_cast` - -## Shapes and Shaping - -TensorFlow provides several operations that you can use to determine the shape -of a tensor and change the shape of a tensor. - -* `tf.broadcast_dynamic_shape` -* `tf.broadcast_static_shape` -* `tf.shape` -* `tf.shape_n` -* `tf.size` -* `tf.rank` -* `tf.reshape` -* `tf.squeeze` -* `tf.expand_dims` -* `tf.meshgrid` - -## Slicing and Joining - -TensorFlow provides several operations to slice or extract parts of a tensor, -or join multiple tensors together. - -* `tf.slice` -* `tf.strided_slice` -* `tf.split` -* `tf.tile` -* `tf.pad` -* `tf.concat` -* `tf.stack` -* `tf.parallel_stack` -* `tf.unstack` -* `tf.reverse_sequence` -* `tf.reverse` -* `tf.reverse_v2` -* `tf.transpose` -* `tf.extract_image_patches` -* `tf.space_to_batch_nd` -* `tf.space_to_batch` -* `tf.required_space_to_batch_paddings` -* `tf.batch_to_space_nd` -* `tf.batch_to_space` -* `tf.space_to_depth` -* `tf.depth_to_space` -* `tf.gather` -* `tf.gather_nd` -* `tf.unique_with_counts` -* `tf.scatter_nd` -* `tf.dynamic_partition` -* `tf.dynamic_stitch` -* `tf.boolean_mask` -* `tf.one_hot` -* `tf.sequence_mask` -* `tf.dequantize` -* `tf.quantize_v2` -* `tf.quantized_concat` -* `tf.setdiff1d` - -## Fake quantization -Operations used to help train for better quantization accuracy. - -* `tf.fake_quant_with_min_max_args` -* `tf.fake_quant_with_min_max_args_gradient` -* `tf.fake_quant_with_min_max_vars` -* `tf.fake_quant_with_min_max_vars_gradient` -* `tf.fake_quant_with_min_max_vars_per_channel` -* `tf.fake_quant_with_min_max_vars_per_channel_gradient` diff --git a/tensorflow/docs_src/api_guides/python/check_ops.md b/tensorflow/docs_src/api_guides/python/check_ops.md deleted file mode 100644 index b52fdaa3ab..0000000000 --- a/tensorflow/docs_src/api_guides/python/check_ops.md +++ /dev/null @@ -1,19 +0,0 @@ -# Asserts and boolean checks - -* `tf.assert_negative` -* `tf.assert_positive` -* `tf.assert_proper_iterable` -* `tf.assert_non_negative` -* `tf.assert_non_positive` -* `tf.assert_equal` -* `tf.assert_integer` -* `tf.assert_less` -* `tf.assert_less_equal` -* `tf.assert_greater` -* `tf.assert_greater_equal` -* `tf.assert_rank` -* `tf.assert_rank_at_least` -* `tf.assert_type` -* `tf.is_non_decreasing` -* `tf.is_numeric_tensor` -* `tf.is_strictly_increasing` diff --git a/tensorflow/docs_src/api_guides/python/client.md b/tensorflow/docs_src/api_guides/python/client.md deleted file mode 100644 index fdd48e66dc..0000000000 --- a/tensorflow/docs_src/api_guides/python/client.md +++ /dev/null @@ -1,36 +0,0 @@ -# Running Graphs -[TOC] - -This library contains classes for launching graphs and executing operations. - -[This guide](../../guide/low_level_intro.md) has examples of how a graph -is launched in a `tf.Session`. - -## Session management - -* `tf.Session` -* `tf.InteractiveSession` -* `tf.get_default_session` - -## Error classes and convenience functions - -* `tf.OpError` -* `tf.errors.CancelledError` -* `tf.errors.UnknownError` -* `tf.errors.InvalidArgumentError` -* `tf.errors.DeadlineExceededError` -* `tf.errors.NotFoundError` -* `tf.errors.AlreadyExistsError` -* `tf.errors.PermissionDeniedError` -* `tf.errors.UnauthenticatedError` -* `tf.errors.ResourceExhaustedError` -* `tf.errors.FailedPreconditionError` -* `tf.errors.AbortedError` -* `tf.errors.OutOfRangeError` -* `tf.errors.UnimplementedError` -* `tf.errors.InternalError` -* `tf.errors.UnavailableError` -* `tf.errors.DataLossError` -* `tf.errors.exception_type_from_error_code` -* `tf.errors.error_code_from_exception_type` -* `tf.errors.raise_exception_on_not_ok_status` diff --git a/tensorflow/docs_src/api_guides/python/constant_op.md b/tensorflow/docs_src/api_guides/python/constant_op.md deleted file mode 100644 index 9ba95b0f55..0000000000 --- a/tensorflow/docs_src/api_guides/python/constant_op.md +++ /dev/null @@ -1,87 +0,0 @@ -# Constants, Sequences, and Random Values - -Note: Functions taking `Tensor` arguments can also take anything accepted by -`tf.convert_to_tensor`. - -[TOC] - -## Constant Value Tensors - -TensorFlow provides several operations that you can use to generate constants. - -* `tf.zeros` -* `tf.zeros_like` -* `tf.ones` -* `tf.ones_like` -* `tf.fill` -* `tf.constant` - -## Sequences - -* `tf.linspace` -* `tf.range` - -## Random Tensors - -TensorFlow has several ops that create random tensors with different -distributions. The random ops are stateful, and create new random values each -time they are evaluated. - -The `seed` keyword argument in these functions acts in conjunction with -the graph-level random seed. Changing either the graph-level seed using -`tf.set_random_seed` or the -op-level seed will change the underlying seed of these operations. Setting -neither graph-level nor op-level seed, results in a random seed for all -operations. -See `tf.set_random_seed` -for details on the interaction between operation-level and graph-level random -seeds. - -### Examples: - -```python -# Create a tensor of shape [2, 3] consisting of random normal values, with mean -# -1 and standard deviation 4. -norm = tf.random_normal([2, 3], mean=-1, stddev=4) - -# Shuffle the first dimension of a tensor -c = tf.constant([[1, 2], [3, 4], [5, 6]]) -shuff = tf.random_shuffle(c) - -# Each time we run these ops, different results are generated -sess = tf.Session() -print(sess.run(norm)) -print(sess.run(norm)) - -# Set an op-level seed to generate repeatable sequences across sessions. -norm = tf.random_normal([2, 3], seed=1234) -sess = tf.Session() -print(sess.run(norm)) -print(sess.run(norm)) -sess = tf.Session() -print(sess.run(norm)) -print(sess.run(norm)) -``` - -Another common use of random values is the initialization of variables. Also see -the [Variables How To](../../guide/variables.md). - -```python -# Use random uniform values in [0, 1) as the initializer for a variable of shape -# [2, 3]. The default type is float32. -var = tf.Variable(tf.random_uniform([2, 3]), name="var") -init = tf.global_variables_initializer() - -sess = tf.Session() -sess.run(init) -print(sess.run(var)) -``` - -* `tf.random_normal` -* `tf.truncated_normal` -* `tf.random_uniform` -* `tf.random_shuffle` -* `tf.random_crop` -* `tf.multinomial` -* `tf.random_gamma` -* `tf.set_random_seed` diff --git a/tensorflow/docs_src/api_guides/python/contrib.crf.md b/tensorflow/docs_src/api_guides/python/contrib.crf.md deleted file mode 100644 index a544f136b3..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.crf.md +++ /dev/null @@ -1,11 +0,0 @@ -# CRF (contrib) - -Linear-chain CRF layer. - -* `tf.contrib.crf.crf_sequence_score` -* `tf.contrib.crf.crf_log_norm` -* `tf.contrib.crf.crf_log_likelihood` -* `tf.contrib.crf.crf_unary_score` -* `tf.contrib.crf.crf_binary_score` -* `tf.contrib.crf.CrfForwardRnnCell` -* `tf.contrib.crf.viterbi_decode` diff --git a/tensorflow/docs_src/api_guides/python/contrib.ffmpeg.md b/tensorflow/docs_src/api_guides/python/contrib.ffmpeg.md deleted file mode 100644 index 7df7547131..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.ffmpeg.md +++ /dev/null @@ -1,23 +0,0 @@ -# FFmpeg (contrib) -[TOC] - -## Encoding and decoding audio using FFmpeg - -TensorFlow provides Ops to decode and encode audio files using the -[FFmpeg](https://www.ffmpeg.org/) library. FFmpeg must be -locally [installed](https://ffmpeg.org/download.html) for these Ops to succeed. - -Example: - -```python -from tensorflow.contrib import ffmpeg - -audio_binary = tf.read_file('song.mp3') -waveform = ffmpeg.decode_audio( - audio_binary, file_format='mp3', samples_per_second=44100, channel_count=2) -uncompressed_binary = ffmpeg.encode_audio( - waveform, file_format='wav', samples_per_second=44100) -``` - -* `tf.contrib.ffmpeg.decode_audio` -* `tf.contrib.ffmpeg.encode_audio` diff --git a/tensorflow/docs_src/api_guides/python/contrib.framework.md b/tensorflow/docs_src/api_guides/python/contrib.framework.md deleted file mode 100644 index 00fb8b0ac3..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.framework.md +++ /dev/null @@ -1,64 +0,0 @@ -# Framework (contrib) -[TOC] - -Framework utilities. - -* `tf.contrib.framework.assert_same_float_dtype` -* `tf.contrib.framework.assert_scalar` -* `tf.contrib.framework.assert_scalar_int` -* `tf.convert_to_tensor_or_sparse_tensor` -* `tf.contrib.framework.get_graph_from_inputs` -* `tf.is_numeric_tensor` -* `tf.is_non_decreasing` -* `tf.is_strictly_increasing` -* `tf.contrib.framework.is_tensor` -* `tf.contrib.framework.reduce_sum_n` -* `tf.contrib.framework.remove_squeezable_dimensions` -* `tf.contrib.framework.with_shape` -* `tf.contrib.framework.with_same_shape` - -## Deprecation - -* `tf.contrib.framework.deprecated` -* `tf.contrib.framework.deprecated_args` -* `tf.contrib.framework.deprecated_arg_values` - -## Arg_Scope - -* `tf.contrib.framework.arg_scope` -* `tf.contrib.framework.add_arg_scope` -* `tf.contrib.framework.has_arg_scope` -* `tf.contrib.framework.arg_scoped_arguments` - -## Variables - -* `tf.contrib.framework.add_model_variable` -* `tf.train.assert_global_step` -* `tf.contrib.framework.assert_or_get_global_step` -* `tf.contrib.framework.assign_from_checkpoint` -* `tf.contrib.framework.assign_from_checkpoint_fn` -* `tf.contrib.framework.assign_from_values` -* `tf.contrib.framework.assign_from_values_fn` -* `tf.contrib.framework.create_global_step` -* `tf.contrib.framework.filter_variables` -* `tf.train.get_global_step` -* `tf.contrib.framework.get_or_create_global_step` -* `tf.contrib.framework.get_local_variables` -* `tf.contrib.framework.get_model_variables` -* `tf.contrib.framework.get_unique_variable` -* `tf.contrib.framework.get_variables_by_name` -* `tf.contrib.framework.get_variables_by_suffix` -* `tf.contrib.framework.get_variables_to_restore` -* `tf.contrib.framework.get_variables` -* `tf.contrib.framework.local_variable` -* `tf.contrib.framework.model_variable` -* `tf.contrib.framework.variable` -* `tf.contrib.framework.VariableDeviceChooser` -* `tf.contrib.framework.zero_initializer` - -## Checkpoint utilities - -* `tf.contrib.framework.load_checkpoint` -* `tf.contrib.framework.list_variables` -* `tf.contrib.framework.load_variable` -* `tf.contrib.framework.init_from_checkpoint` diff --git a/tensorflow/docs_src/api_guides/python/contrib.graph_editor.md b/tensorflow/docs_src/api_guides/python/contrib.graph_editor.md deleted file mode 100644 index 8ce49b952b..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.graph_editor.md +++ /dev/null @@ -1,177 +0,0 @@ -# Graph Editor (contrib) -[TOC] - -TensorFlow Graph Editor. - -The TensorFlow Graph Editor library allows for modification of an existing -`tf.Graph` instance in-place. - -The author's github username is [purpledog](https://github.com/purpledog). - -## Library overview - -Appending new nodes is the only graph editing operation allowed by the -TensorFlow core library. The Graph Editor library is an attempt to allow for -other kinds of editing operations, namely, *rerouting* and *transforming*. - -* *rerouting* is a local operation consisting in re-plugging existing tensors - (the edges of the graph). Operations (the nodes) are not modified by this - operation. For example, rerouting can be used to insert an operation adding - noise in place of an existing tensor. -* *transforming* is a global operation consisting in transforming a graph into - another. By default, a transformation is a simple copy but it can be - customized to achieved other goals. For instance, a graph can be transformed - into another one in which noise is added after all the operations of a - specific type. - -**Important: modifying a graph in-place with the Graph Editor must be done -`offline`, that is, without any active sessions.** - -Of course new operations can be appended online but Graph Editor specific -operations like rerouting and transforming can currently only be done offline. - -Here is an example of what you **cannot** do: - -* Build a graph. -* Create a session and run the graph. -* Modify the graph with the Graph Editor. -* Re-run the graph with the `same` previously created session. - -To edit an already running graph, follow these steps: - -* Build a graph. -* Create a session and run the graph. -* Save the graph state and terminate the session -* Modify the graph with the Graph Editor. -* create a new session and restore the graph state -* Re-run the graph with the newly created session. - -Note that this procedure is very costly because a new session must be created -after any modifications. Among other things, it takes time because the entire -graph state must be saved and restored again. - -## Sub-graph - -Most of the functions in the Graph Editor library operate on *sub-graph*. -More precisely, they take as input arguments instances of the SubGraphView class -(or anything which can be converted to it). Doing so allows the same function -to transparently operate on single operations as well as sub-graph of any size. - -A subgraph can be created in several ways: - -* using a list of ops: - - ```python - my_sgv = ge.sgv(ops) - ``` - -* from a name scope: - - ```python - my_sgv = ge.sgv_scope("foo/bar", graph=tf.get_default_graph()) - ``` - -* using regular expression: - - ```python - my_sgv = ge.sgv("foo/.*/.*read$", graph=tf.get_default_graph()) - ``` - -Note that the Graph Editor is meant to manipulate several graphs at the same -time, typically during transform or copy operation. For that reason, -to avoid any confusion, the default graph is never used and the graph on -which to operate must always be given explicitly. This is the reason why -*`graph=tf.get_default_graph()`* is used in the code snippets above. - -## Modules overview - -* util: utility functions. -* select: various selection methods of TensorFlow tensors and operations. -* match: TensorFlow graph matching. Think of this as regular expressions for - graphs (but not quite yet). -* reroute: various ways of rerouting tensors to different consuming ops like - *swap* or *reroute_a2b*. -* subgraph: the SubGraphView class, which enables subgraph manipulations in a - TensorFlow `tf.Graph`. -* edit: various editing functions operating on subgraphs like *detach*, - *connect* or *bypass*. -* transform: the Transformer class, which enables transforming - (or simply copying) a subgraph into another one. - -## Module: util - -* `tf.contrib.graph_editor.make_list_of_op` -* `tf.contrib.graph_editor.get_tensors` -* `tf.contrib.graph_editor.make_list_of_t` -* `tf.contrib.graph_editor.get_generating_ops` -* `tf.contrib.graph_editor.get_consuming_ops` -* `tf.contrib.graph_editor.ControlOutputs` -* `tf.contrib.graph_editor.placeholder_name` -* `tf.contrib.graph_editor.make_placeholder_from_tensor` -* `tf.contrib.graph_editor.make_placeholder_from_dtype_and_shape` - -## Module: select - -* `tf.contrib.graph_editor.filter_ts` -* `tf.contrib.graph_editor.filter_ts_from_regex` -* `tf.contrib.graph_editor.filter_ops` -* `tf.contrib.graph_editor.filter_ops_from_regex` -* `tf.contrib.graph_editor.get_name_scope_ops` -* `tf.contrib.graph_editor.check_cios` -* `tf.contrib.graph_editor.get_ops_ios` -* `tf.contrib.graph_editor.compute_boundary_ts` -* `tf.contrib.graph_editor.get_within_boundary_ops` -* `tf.contrib.graph_editor.get_forward_walk_ops` -* `tf.contrib.graph_editor.get_backward_walk_ops` -* `tf.contrib.graph_editor.get_walks_intersection_ops` -* `tf.contrib.graph_editor.get_walks_union_ops` -* `tf.contrib.graph_editor.select_ops` -* `tf.contrib.graph_editor.select_ts` -* `tf.contrib.graph_editor.select_ops_and_ts` - -## Module: subgraph - -* `tf.contrib.graph_editor.SubGraphView` -* `tf.contrib.graph_editor.make_view` -* `tf.contrib.graph_editor.make_view_from_scope` - -## Module: reroute - -* `tf.contrib.graph_editor.swap_ts` -* `tf.contrib.graph_editor.reroute_ts` -* `tf.contrib.graph_editor.swap_inputs` -* `tf.contrib.graph_editor.reroute_inputs` -* `tf.contrib.graph_editor.swap_outputs` -* `tf.contrib.graph_editor.reroute_outputs` -* `tf.contrib.graph_editor.swap_ios` -* `tf.contrib.graph_editor.reroute_ios` -* `tf.contrib.graph_editor.remove_control_inputs` -* `tf.contrib.graph_editor.add_control_inputs` - -## Module: edit - -* `tf.contrib.graph_editor.detach_control_inputs` -* `tf.contrib.graph_editor.detach_control_outputs` -* `tf.contrib.graph_editor.detach_inputs` -* `tf.contrib.graph_editor.detach_outputs` -* `tf.contrib.graph_editor.detach` -* `tf.contrib.graph_editor.connect` -* `tf.contrib.graph_editor.bypass` - -## Module: transform - -* `tf.contrib.graph_editor.replace_t_with_placeholder_handler` -* `tf.contrib.graph_editor.keep_t_if_possible_handler` -* `tf.contrib.graph_editor.assign_renamed_collections_handler` -* `tf.contrib.graph_editor.transform_op_if_inside_handler` -* `tf.contrib.graph_editor.copy_op_handler` -* `tf.contrib.graph_editor.Transformer` -* `tf.contrib.graph_editor.copy` -* `tf.contrib.graph_editor.copy_with_input_replacements` -* `tf.contrib.graph_editor.graph_replace` - -## Useful aliases - -* `tf.contrib.graph_editor.ph` -* `tf.contrib.graph_editor.sgv` -* `tf.contrib.graph_editor.sgv_scope` diff --git a/tensorflow/docs_src/api_guides/python/contrib.integrate.md b/tensorflow/docs_src/api_guides/python/contrib.integrate.md deleted file mode 100644 index a70d202ab5..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.integrate.md +++ /dev/null @@ -1,41 +0,0 @@ -# Integrate (contrib) -[TOC] - -Integration and ODE solvers for TensorFlow. - -## Example: Lorenz attractor - -We can use `odeint` to solve the -[Lorentz system](https://en.wikipedia.org/wiki/Lorenz_system) of ordinary -differential equations, a prototypical example of chaotic dynamics: - -```python -rho = 28.0 -sigma = 10.0 -beta = 8.0/3.0 - -def lorenz_equation(state, t): - x, y, z = tf.unstack(state) - dx = sigma * (y - x) - dy = x * (rho - z) - y - dz = x * y - beta * z - return tf.stack([dx, dy, dz]) - -init_state = tf.constant([0, 2, 20], dtype=tf.float64) -t = np.linspace(0, 50, num=5000) -tensor_state, tensor_info = tf.contrib.integrate.odeint( - lorenz_equation, init_state, t, full_output=True) - -sess = tf.Session() -state, info = sess.run([tensor_state, tensor_info]) -x, y, z = state.T -plt.plot(x, z) -``` - -<div style="width:70%; margin:auto; margin-bottom:10px; margin-top:20px;"> -<img style="width:100%" src="https://www.tensorflow.org/images/lorenz_attractor.png" alt> -</div> - -## Ops - -* `tf.contrib.integrate.odeint` diff --git a/tensorflow/docs_src/api_guides/python/contrib.layers.md b/tensorflow/docs_src/api_guides/python/contrib.layers.md deleted file mode 100644 index 4c176a129c..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.layers.md +++ /dev/null @@ -1,109 +0,0 @@ -# Layers (contrib) -[TOC] - -Ops for building neural network layers, regularizers, summaries, etc. - -## Higher level ops for building neural network layers - -This package provides several ops that take care of creating variables that are -used internally in a consistent way and provide the building blocks for many -common machine learning algorithms. - -* `tf.contrib.layers.avg_pool2d` -* `tf.contrib.layers.batch_norm` -* `tf.contrib.layers.convolution2d` -* `tf.contrib.layers.conv2d_in_plane` -* `tf.contrib.layers.convolution2d_in_plane` -* `tf.nn.conv2d_transpose` -* `tf.contrib.layers.convolution2d_transpose` -* `tf.nn.dropout` -* `tf.contrib.layers.flatten` -* `tf.contrib.layers.fully_connected` -* `tf.contrib.layers.layer_norm` -* `tf.contrib.layers.max_pool2d` -* `tf.contrib.layers.one_hot_encoding` -* `tf.nn.relu` -* `tf.nn.relu6` -* `tf.contrib.layers.repeat` -* `tf.contrib.layers.safe_embedding_lookup_sparse` -* `tf.nn.separable_conv2d` -* `tf.contrib.layers.separable_convolution2d` -* `tf.nn.softmax` -* `tf.stack` -* `tf.contrib.layers.unit_norm` -* `tf.contrib.layers.embed_sequence` - -Aliases for fully_connected which set a default activation function are -available: `relu`, `relu6` and `linear`. - -`stack` operation is also available. It builds a stack of layers by applying -a layer repeatedly. - -## Regularizers - -Regularization can help prevent overfitting. These have the signature -`fn(weights)`. The loss is typically added to -`tf.GraphKeys.REGULARIZATION_LOSSES`. - -* `tf.contrib.layers.apply_regularization` -* `tf.contrib.layers.l1_regularizer` -* `tf.contrib.layers.l2_regularizer` -* `tf.contrib.layers.sum_regularizer` - -## Initializers - -Initializers are used to initialize variables with sensible values given their -size, data type, and purpose. - -* `tf.contrib.layers.xavier_initializer` -* `tf.contrib.layers.xavier_initializer_conv2d` -* `tf.contrib.layers.variance_scaling_initializer` - -## Optimization - -Optimize weights given a loss. - -* `tf.contrib.layers.optimize_loss` - -## Summaries - -Helper functions to summarize specific variables or ops. - -* `tf.contrib.layers.summarize_activation` -* `tf.contrib.layers.summarize_tensor` -* `tf.contrib.layers.summarize_tensors` -* `tf.contrib.layers.summarize_collection` - -The layers module defines convenience functions `summarize_variables`, -`summarize_weights` and `summarize_biases`, which set the `collection` argument -of `summarize_collection` to `VARIABLES`, `WEIGHTS` and `BIASES`, respectively. - -* `tf.contrib.layers.summarize_activations` - -## Feature columns - -Feature columns provide a mechanism to map data to a model. - -* `tf.contrib.layers.bucketized_column` -* `tf.contrib.layers.check_feature_columns` -* `tf.contrib.layers.create_feature_spec_for_parsing` -* `tf.contrib.layers.crossed_column` -* `tf.contrib.layers.embedding_column` -* `tf.contrib.layers.scattered_embedding_column` -* `tf.contrib.layers.input_from_feature_columns` -* `tf.contrib.layers.joint_weighted_sum_from_feature_columns` -* `tf.contrib.layers.make_place_holder_tensors_for_base_features` -* `tf.contrib.layers.multi_class_target` -* `tf.contrib.layers.one_hot_column` -* `tf.contrib.layers.parse_feature_columns_from_examples` -* `tf.contrib.layers.parse_feature_columns_from_sequence_examples` -* `tf.contrib.layers.real_valued_column` -* `tf.contrib.layers.shared_embedding_columns` -* `tf.contrib.layers.sparse_column_with_hash_bucket` -* `tf.contrib.layers.sparse_column_with_integerized_feature` -* `tf.contrib.layers.sparse_column_with_keys` -* `tf.contrib.layers.sparse_column_with_vocabulary_file` -* `tf.contrib.layers.weighted_sparse_column` -* `tf.contrib.layers.weighted_sum_from_feature_columns` -* `tf.contrib.layers.infer_real_valued_columns` -* `tf.contrib.layers.sequence_input_from_feature_columns` diff --git a/tensorflow/docs_src/api_guides/python/contrib.learn.md b/tensorflow/docs_src/api_guides/python/contrib.learn.md deleted file mode 100644 index 635849ead5..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.learn.md +++ /dev/null @@ -1,63 +0,0 @@ -# Learn (contrib) -[TOC] - -High level API for learning with TensorFlow. - -## Estimators - -Train and evaluate TensorFlow models. - -* `tf.contrib.learn.BaseEstimator` -* `tf.contrib.learn.Estimator` -* `tf.contrib.learn.Trainable` -* `tf.contrib.learn.Evaluable` -* `tf.contrib.learn.KMeansClustering` -* `tf.contrib.learn.ModeKeys` -* `tf.contrib.learn.ModelFnOps` -* `tf.contrib.learn.MetricSpec` -* `tf.contrib.learn.PredictionKey` -* `tf.contrib.learn.DNNClassifier` -* `tf.contrib.learn.DNNRegressor` -* `tf.contrib.learn.DNNLinearCombinedRegressor` -* `tf.contrib.learn.DNNLinearCombinedClassifier` -* `tf.contrib.learn.LinearClassifier` -* `tf.contrib.learn.LinearRegressor` -* `tf.contrib.learn.LogisticRegressor` - -## Distributed training utilities - -* `tf.contrib.learn.Experiment` -* `tf.contrib.learn.ExportStrategy` -* `tf.contrib.learn.TaskType` - -## Graph actions - -Perform various training, evaluation, and inference actions on a graph. - -* `tf.train.NanLossDuringTrainingError` -* `tf.contrib.learn.RunConfig` -* `tf.contrib.learn.evaluate` -* `tf.contrib.learn.infer` -* `tf.contrib.learn.run_feeds` -* `tf.contrib.learn.run_n` -* `tf.contrib.learn.train` - -## Input processing - -Queue and read batched input data. - -* `tf.contrib.learn.extract_dask_data` -* `tf.contrib.learn.extract_dask_labels` -* `tf.contrib.learn.extract_pandas_data` -* `tf.contrib.learn.extract_pandas_labels` -* `tf.contrib.learn.extract_pandas_matrix` -* `tf.contrib.learn.infer_real_valued_columns_from_input` -* `tf.contrib.learn.infer_real_valued_columns_from_input_fn` -* `tf.contrib.learn.read_batch_examples` -* `tf.contrib.learn.read_batch_features` -* `tf.contrib.learn.read_batch_record_features` - -Export utilities - -* `tf.contrib.learn.build_parsing_serving_input_fn` -* `tf.contrib.learn.ProblemType` diff --git a/tensorflow/docs_src/api_guides/python/contrib.linalg.md b/tensorflow/docs_src/api_guides/python/contrib.linalg.md deleted file mode 100644 index 3055449dc2..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.linalg.md +++ /dev/null @@ -1,30 +0,0 @@ -# Linear Algebra (contrib) -[TOC] - -Linear algebra libraries for TensorFlow. - -## `LinearOperator` - -Subclasses of `LinearOperator` provide a access to common methods on a -(batch) matrix, without the need to materialize the matrix. This allows: - -* Matrix free computations -* Different operators to take advantage of special structure, while providing a - consistent API to users. - -### Base class - -* `tf.contrib.linalg.LinearOperator` - -### Individual operators - -* `tf.contrib.linalg.LinearOperatorDiag` -* `tf.contrib.linalg.LinearOperatorIdentity` -* `tf.contrib.linalg.LinearOperatorScaledIdentity` -* `tf.contrib.linalg.LinearOperatorFullMatrix` -* `tf.contrib.linalg.LinearOperatorLowerTriangular` -* `tf.contrib.linalg.LinearOperatorLowRankUpdate` - -### Transformations and Combinations of operators - -* `tf.contrib.linalg.LinearOperatorComposition` diff --git a/tensorflow/docs_src/api_guides/python/contrib.losses.md b/tensorflow/docs_src/api_guides/python/contrib.losses.md deleted file mode 100644 index 8787454af6..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.losses.md +++ /dev/null @@ -1,125 +0,0 @@ -# Losses (contrib) - -## Deprecated - -This module is deprecated. Instructions for updating: Use `tf.losses` instead. - -## Loss operations for use in neural networks. - -Note: By default, all the losses are collected into the `GraphKeys.LOSSES` -collection. - -All of the loss functions take a pair of predictions and ground truth labels, -from which the loss is computed. It is assumed that the shape of both these -tensors is of the form [batch_size, d1, ... dN] where `batch_size` is the number -of samples in the batch and `d1` ... `dN` are the remaining dimensions. - -It is common, when training with multiple loss functions, to adjust the relative -strengths of individual losses. This is performed by rescaling the losses via -a `weight` parameter passed to the loss functions. For example, if we were -training with both log_loss and mean_squared_error, and we wished that the -log_loss penalty be twice as severe as the mean_squared_error, we would -implement this as: - -```python - # Explicitly set the weight. - tf.contrib.losses.log(predictions, labels, weight=2.0) - - # Uses default weight of 1.0 - tf.contrib.losses.mean_squared_error(predictions, labels) - - # All the losses are collected into the `GraphKeys.LOSSES` collection. - losses = tf.get_collection(tf.GraphKeys.LOSSES) -``` - -While specifying a scalar loss rescales the loss over the entire batch, -we sometimes want to rescale the loss per batch sample. For example, if we have -certain examples that matter more to us to get correctly, we might want to have -a higher loss that other samples whose mistakes matter less. In this case, we -can provide a weight vector of length `batch_size` which results in the loss -for each sample in the batch being scaled by the corresponding weight element. -For example, consider the case of a classification problem where we want to -maximize our accuracy but we especially interested in obtaining high accuracy -for a specific class: - -```python - inputs, labels = LoadData(batch_size=3) - logits = MyModelPredictions(inputs) - - # Ensures that the loss for examples whose ground truth class is `3` is 5x - # higher than the loss for all other examples. - weight = tf.multiply(4, tf.cast(tf.equal(labels, 3), tf.float32)) + 1 - - onehot_labels = tf.one_hot(labels, num_classes=5) - tf.contrib.losses.softmax_cross_entropy(logits, onehot_labels, weight=weight) -``` - -Finally, in certain cases, we may want to specify a different loss for every -single measurable value. For example, if we are performing per-pixel depth -prediction, or per-pixel denoising, a single batch sample has P values where P -is the number of pixels in the image. For many losses, the number of measurable -values matches the number of elements in the predictions and labels tensors. -For others, such as softmax_cross_entropy and cosine_distance, the -loss functions reduces the dimensions of the inputs to produces a tensor of -losses for each measurable value. For example, softmax_cross_entropy takes as -input predictions and labels of dimension [batch_size, num_classes] but the -number of measurable values is [batch_size]. Consequently, when passing a weight -tensor to specify a different loss for every measurable value, the dimension of -the tensor will depend on the loss being used. - -For a concrete example, consider the case of per-pixel depth prediction where -certain ground truth depth values are missing (due to sensor noise in the -capture process). In this case, we want to assign zero weight to losses for -these predictions. - -```python - # 'depths' that are missing have a value of 0: - images, depths = LoadData(...) - predictions = MyModelPredictions(images) - - weight = tf.cast(tf.greater(depths, 0), tf.float32) - loss = tf.contrib.losses.mean_squared_error(predictions, depths, weight) -``` - -Note that when using weights for the losses, the final average is computed -by rescaling the losses by the weights and then dividing by the total number of -non-zero samples. For an arbitrary set of weights, this may not necessarily -produce a weighted average. Instead, it simply and transparently rescales the -per-element losses before averaging over the number of observations. For example -if the losses computed by the loss function is an array [4, 1, 2, 3] and the -weights are an array [1, 0.5, 3, 9], then the average loss is: - -```python - (4*1 + 1*0.5 + 2*3 + 3*9) / 4 -``` - -However, with a single loss function and an arbitrary set of weights, one can -still easily create a loss function such that the resulting loss is a -weighted average over the individual prediction errors: - - -```python - images, labels = LoadData(...) - predictions = MyModelPredictions(images) - - weight = MyComplicatedWeightingFunction(labels) - weight = tf.div(weight, tf.size(weight)) - loss = tf.contrib.losses.mean_squared_error(predictions, depths, weight) -``` - -* `tf.contrib.losses.absolute_difference` -* `tf.contrib.losses.add_loss` -* `tf.contrib.losses.hinge_loss` -* `tf.contrib.losses.compute_weighted_loss` -* `tf.contrib.losses.cosine_distance` -* `tf.contrib.losses.get_losses` -* `tf.contrib.losses.get_regularization_losses` -* `tf.contrib.losses.get_total_loss` -* `tf.contrib.losses.log_loss` -* `tf.contrib.losses.mean_pairwise_squared_error` -* `tf.contrib.losses.mean_squared_error` -* `tf.contrib.losses.sigmoid_cross_entropy` -* `tf.contrib.losses.softmax_cross_entropy` -* `tf.contrib.losses.sparse_softmax_cross_entropy` - - diff --git a/tensorflow/docs_src/api_guides/python/contrib.metrics.md b/tensorflow/docs_src/api_guides/python/contrib.metrics.md deleted file mode 100644 index de6346ca80..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.metrics.md +++ /dev/null @@ -1,133 +0,0 @@ -# Metrics (contrib) -[TOC] - -##Ops for evaluation metrics and summary statistics. - -### API - -This module provides functions for computing streaming metrics: metrics computed -on dynamically valued `Tensors`. Each metric declaration returns a -"value_tensor", an idempotent operation that returns the current value of the -metric, and an "update_op", an operation that accumulates the information -from the current value of the `Tensors` being measured as well as returns the -value of the "value_tensor". - -To use any of these metrics, one need only declare the metric, call `update_op` -repeatedly to accumulate data over the desired number of `Tensor` values (often -each one is a single batch) and finally evaluate the value_tensor. For example, -to use the `streaming_mean`: - -```python -value = ... -mean_value, update_op = tf.contrib.metrics.streaming_mean(values) -sess.run(tf.local_variables_initializer()) - -for i in range(number_of_batches): - print('Mean after batch %d: %f' % (i, update_op.eval()) -print('Final Mean: %f' % mean_value.eval()) -``` - -Each metric function adds nodes to the graph that hold the state necessary to -compute the value of the metric as well as a set of operations that actually -perform the computation. Every metric evaluation is composed of three steps - -* Initialization: initializing the metric state. -* Aggregation: updating the values of the metric state. -* Finalization: computing the final metric value. - -In the above example, calling streaming_mean creates a pair of state variables -that will contain (1) the running sum and (2) the count of the number of samples -in the sum. Because the streaming metrics use local variables, -the Initialization stage is performed by running the op returned -by `tf.local_variables_initializer()`. It sets the sum and count variables to -zero. - -Next, Aggregation is performed by examining the current state of `values` -and incrementing the state variables appropriately. This step is executed by -running the `update_op` returned by the metric. - -Finally, finalization is performed by evaluating the "value_tensor" - -In practice, we commonly want to evaluate across many batches and multiple -metrics. To do so, we need only run the metric computation operations multiple -times: - -```python -labels = ... -predictions = ... -accuracy, update_op_acc = tf.contrib.metrics.streaming_accuracy( - labels, predictions) -error, update_op_error = tf.contrib.metrics.streaming_mean_absolute_error( - labels, predictions) - -sess.run(tf.local_variables_initializer()) -for batch in range(num_batches): - sess.run([update_op_acc, update_op_error]) - -accuracy, error = sess.run([accuracy, error]) -``` - -Note that when evaluating the same metric multiple times on different inputs, -one must specify the scope of each metric to avoid accumulating the results -together: - -```python -labels = ... -predictions0 = ... -predictions1 = ... - -accuracy0 = tf.contrib.metrics.accuracy(labels, predictions0, name='preds0') -accuracy1 = tf.contrib.metrics.accuracy(labels, predictions1, name='preds1') -``` - -Certain metrics, such as streaming_mean or streaming_accuracy, can be weighted -via a `weights` argument. The `weights` tensor must be the same size as the -labels and predictions tensors and results in a weighted average of the metric. - -## Metric `Ops` - -* `tf.contrib.metrics.streaming_accuracy` -* `tf.contrib.metrics.streaming_mean` -* `tf.contrib.metrics.streaming_recall` -* `tf.contrib.metrics.streaming_recall_at_thresholds` -* `tf.contrib.metrics.streaming_precision` -* `tf.contrib.metrics.streaming_precision_at_thresholds` -* `tf.contrib.metrics.streaming_auc` -* `tf.contrib.metrics.streaming_recall_at_k` -* `tf.contrib.metrics.streaming_mean_absolute_error` -* `tf.contrib.metrics.streaming_mean_iou` -* `tf.contrib.metrics.streaming_mean_relative_error` -* `tf.contrib.metrics.streaming_mean_squared_error` -* `tf.contrib.metrics.streaming_mean_tensor` -* `tf.contrib.metrics.streaming_root_mean_squared_error` -* `tf.contrib.metrics.streaming_covariance` -* `tf.contrib.metrics.streaming_pearson_correlation` -* `tf.contrib.metrics.streaming_mean_cosine_distance` -* `tf.contrib.metrics.streaming_percentage_less` -* `tf.contrib.metrics.streaming_sensitivity_at_specificity` -* `tf.contrib.metrics.streaming_sparse_average_precision_at_k` -* `tf.contrib.metrics.streaming_sparse_precision_at_k` -* `tf.contrib.metrics.streaming_sparse_precision_at_top_k` -* `tf.contrib.metrics.streaming_sparse_recall_at_k` -* `tf.contrib.metrics.streaming_specificity_at_sensitivity` -* `tf.contrib.metrics.streaming_concat` -* `tf.contrib.metrics.streaming_false_negatives` -* `tf.contrib.metrics.streaming_false_negatives_at_thresholds` -* `tf.contrib.metrics.streaming_false_positives` -* `tf.contrib.metrics.streaming_false_positives_at_thresholds` -* `tf.contrib.metrics.streaming_true_negatives` -* `tf.contrib.metrics.streaming_true_negatives_at_thresholds` -* `tf.contrib.metrics.streaming_true_positives` -* `tf.contrib.metrics.streaming_true_positives_at_thresholds` -* `tf.contrib.metrics.auc_using_histogram` -* `tf.contrib.metrics.accuracy` -* `tf.contrib.metrics.aggregate_metrics` -* `tf.contrib.metrics.aggregate_metric_map` -* `tf.contrib.metrics.confusion_matrix` - -## Set `Ops` - -* `tf.contrib.metrics.set_difference` -* `tf.contrib.metrics.set_intersection` -* `tf.contrib.metrics.set_size` -* `tf.contrib.metrics.set_union` diff --git a/tensorflow/docs_src/api_guides/python/contrib.rnn.md b/tensorflow/docs_src/api_guides/python/contrib.rnn.md deleted file mode 100644 index d265ab6925..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.rnn.md +++ /dev/null @@ -1,61 +0,0 @@ -# RNN and Cells (contrib) -[TOC] - -Module for constructing RNN Cells and additional RNN operations. - -## Base interface for all RNN Cells - -* `tf.contrib.rnn.RNNCell` - -## Core RNN Cells for use with TensorFlow's core RNN methods - -* `tf.contrib.rnn.BasicRNNCell` -* `tf.contrib.rnn.BasicLSTMCell` -* `tf.contrib.rnn.GRUCell` -* `tf.contrib.rnn.LSTMCell` -* `tf.contrib.rnn.LayerNormBasicLSTMCell` - -## Classes storing split `RNNCell` state - -* `tf.contrib.rnn.LSTMStateTuple` - -## Core RNN Cell wrappers (RNNCells that wrap other RNNCells) - -* `tf.contrib.rnn.MultiRNNCell` -* `tf.contrib.rnn.LSTMBlockWrapper` -* `tf.contrib.rnn.DropoutWrapper` -* `tf.contrib.rnn.EmbeddingWrapper` -* `tf.contrib.rnn.InputProjectionWrapper` -* `tf.contrib.rnn.OutputProjectionWrapper` -* `tf.contrib.rnn.DeviceWrapper` -* `tf.contrib.rnn.ResidualWrapper` - -### Block RNNCells -* `tf.contrib.rnn.LSTMBlockCell` -* `tf.contrib.rnn.GRUBlockCell` - -### Fused RNNCells -* `tf.contrib.rnn.FusedRNNCell` -* `tf.contrib.rnn.FusedRNNCellAdaptor` -* `tf.contrib.rnn.TimeReversedFusedRNN` -* `tf.contrib.rnn.LSTMBlockFusedCell` - -### LSTM-like cells -* `tf.contrib.rnn.CoupledInputForgetGateLSTMCell` -* `tf.contrib.rnn.TimeFreqLSTMCell` -* `tf.contrib.rnn.GridLSTMCell` - -### RNNCell wrappers -* `tf.contrib.rnn.AttentionCellWrapper` -* `tf.contrib.rnn.CompiledWrapper` - - -## Recurrent Neural Networks - -TensorFlow provides a number of methods for constructing Recurrent Neural -Networks. - -* `tf.contrib.rnn.static_rnn` -* `tf.contrib.rnn.static_state_saving_rnn` -* `tf.contrib.rnn.static_bidirectional_rnn` -* `tf.contrib.rnn.stack_bidirectional_dynamic_rnn` diff --git a/tensorflow/docs_src/api_guides/python/contrib.seq2seq.md b/tensorflow/docs_src/api_guides/python/contrib.seq2seq.md deleted file mode 100644 index 54f2fafc71..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.seq2seq.md +++ /dev/null @@ -1,138 +0,0 @@ -# Seq2seq Library (contrib) -[TOC] - -Module for constructing seq2seq models and dynamic decoding. Builds on top of -libraries in `tf.contrib.rnn`. - -This library is composed of two primary components: - -* New attention wrappers for `tf.contrib.rnn.RNNCell` objects. -* A new object-oriented dynamic decoding framework. - -## Attention - -Attention wrappers are `RNNCell` objects that wrap other `RNNCell` objects and -implement attention. The form of attention is determined by a subclass of -`tf.contrib.seq2seq.AttentionMechanism`. These subclasses describe the form -of attention (e.g. additive vs. multiplicative) to use when creating the -wrapper. An instance of an `AttentionMechanism` is constructed with a -`memory` tensor, from which lookup keys and values tensors are created. - -### Attention Mechanisms - -The two basic attention mechanisms are: - -* `tf.contrib.seq2seq.BahdanauAttention` (additive attention, - [ref.](https://arxiv.org/abs/1409.0473)) -* `tf.contrib.seq2seq.LuongAttention` (multiplicative attention, - [ref.](https://arxiv.org/abs/1508.04025)) - -The `memory` tensor passed the attention mechanism's constructor is expected to -be shaped `[batch_size, memory_max_time, memory_depth]`; and often an additional -`memory_sequence_length` vector is accepted. If provided, the `memory` -tensors' rows are masked with zeros past their true sequence lengths. - -Attention mechanisms also have a concept of depth, usually determined as a -construction parameter `num_units`. For some kinds of attention (like -`BahdanauAttention`), both queries and memory are projected to tensors of depth -`num_units`. For other kinds (like `LuongAttention`), `num_units` should match -the depth of the queries; and the `memory` tensor will be projected to this -depth. - -### Attention Wrappers - -The basic attention wrapper is `tf.contrib.seq2seq.AttentionWrapper`. -This wrapper accepts an `RNNCell` instance, an instance of `AttentionMechanism`, -and an attention depth parameter (`attention_size`); as well as several -optional arguments that allow one to customize intermediate calculations. - -At each time step, the basic calculation performed by this wrapper is: - -```python -cell_inputs = concat([inputs, prev_state.attention], -1) -cell_output, next_cell_state = cell(cell_inputs, prev_state.cell_state) -score = attention_mechanism(cell_output) -alignments = softmax(score) -context = matmul(alignments, attention_mechanism.values) -attention = tf.layers.Dense(attention_size)(concat([cell_output, context], 1)) -next_state = AttentionWrapperState( - cell_state=next_cell_state, - attention=attention) -output = attention -return output, next_state -``` - -In practice, a number of the intermediate calculations are configurable. -For example, the initial concatenation of `inputs` and `prev_state.attention` -can be replaced with another mixing function. The function `softmax` can -be replaced with alternative options when calculating `alignments` from the -`score`. Finally, the outputs returned by the wrapper can be configured to -be the value `cell_output` instead of `attention`. - -The benefit of using a `AttentionWrapper` is that it plays nicely with -other wrappers and the dynamic decoder described below. For example, one can -write: - -```python -cell = tf.contrib.rnn.DeviceWrapper(LSTMCell(512), "/device:GPU:0") -attention_mechanism = tf.contrib.seq2seq.LuongAttention(512, encoder_outputs) -attn_cell = tf.contrib.seq2seq.AttentionWrapper( - cell, attention_mechanism, attention_size=256) -attn_cell = tf.contrib.rnn.DeviceWrapper(attn_cell, "/device:GPU:1") -top_cell = tf.contrib.rnn.DeviceWrapper(LSTMCell(512), "/device:GPU:1") -multi_cell = MultiRNNCell([attn_cell, top_cell]) -``` - -The `multi_rnn` cell will perform the bottom layer calculations on GPU 0; -attention calculations will be performed on GPU 1 and immediately passed -up to the top layer which is also calculated on GPU 1. The attention is -also passed forward in time to the next time step and copied to GPU 0 for the -next time step of `cell`. (*Note*: This is just an example of use, -not a suggested device partitioning strategy.) - -## Dynamic Decoding - -Example usage: - -``` python -cell = # instance of RNNCell - -if mode == "train": - helper = tf.contrib.seq2seq.TrainingHelper( - input=input_vectors, - sequence_length=input_lengths) -elif mode == "infer": - helper = tf.contrib.seq2seq.GreedyEmbeddingHelper( - embedding=embedding, - start_tokens=tf.tile([GO_SYMBOL], [batch_size]), - end_token=END_SYMBOL) - -decoder = tf.contrib.seq2seq.BasicDecoder( - cell=cell, - helper=helper, - initial_state=cell.zero_state(batch_size, tf.float32)) -outputs, _ = tf.contrib.seq2seq.dynamic_decode( - decoder=decoder, - output_time_major=False, - impute_finished=True, - maximum_iterations=20) -``` - -### Decoder base class and functions - -* `tf.contrib.seq2seq.Decoder` -* `tf.contrib.seq2seq.dynamic_decode` - -### Basic Decoder - -* `tf.contrib.seq2seq.BasicDecoderOutput` -* `tf.contrib.seq2seq.BasicDecoder` - -### Decoder Helpers - -* `tf.contrib.seq2seq.Helper` -* `tf.contrib.seq2seq.CustomHelper` -* `tf.contrib.seq2seq.GreedyEmbeddingHelper` -* `tf.contrib.seq2seq.ScheduledEmbeddingTrainingHelper` -* `tf.contrib.seq2seq.ScheduledOutputTrainingHelper` -* `tf.contrib.seq2seq.TrainingHelper` diff --git a/tensorflow/docs_src/api_guides/python/contrib.signal.md b/tensorflow/docs_src/api_guides/python/contrib.signal.md deleted file mode 100644 index 66df561084..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.signal.md +++ /dev/null @@ -1,172 +0,0 @@ -# Signal Processing (contrib) -[TOC] - -`tf.contrib.signal` is a module for signal processing primitives. All -operations have GPU support and are differentiable. This module is especially -helpful for building TensorFlow models that process or generate audio, though -the techniques are useful in many domains. - -## Framing variable length sequences - -When dealing with variable length signals (e.g. audio) it is common to "frame" -them into multiple fixed length windows. These windows can overlap if the 'step' -of the frame is less than the frame length. `tf.contrib.signal.frame` does -exactly this. For example: - -```python -# A batch of float32 time-domain signals in the range [-1, 1] with shape -# [batch_size, signal_length]. Both batch_size and signal_length may be unknown. -signals = tf.placeholder(tf.float32, [None, None]) - -# Compute a [batch_size, ?, 128] tensor of fixed length, overlapping windows -# where each window overlaps the previous by 75% (frame_length - frame_step -# samples of overlap). -frames = tf.contrib.signal.frame(signals, frame_length=128, frame_step=32) -``` - -The `axis` parameter to `tf.contrib.signal.frame` allows you to frame tensors -with inner structure (e.g. a spectrogram): - -```python -# `magnitude_spectrograms` is a [batch_size, ?, 129] tensor of spectrograms. We -# would like to produce overlapping fixed-size spectrogram patches; for example, -# for use in a situation where a fixed size input is needed. -magnitude_spectrograms = tf.abs(tf.contrib.signal.stft( - signals, frame_length=256, frame_step=64, fft_length=256)) - -# `spectrogram_patches` is a [batch_size, ?, 64, 129] tensor containing a -# variable number of [64, 129] spectrogram patches per batch item. -spectrogram_patches = tf.contrib.signal.frame( - magnitude_spectrograms, frame_length=64, frame_step=16, axis=1) -``` - -## Reconstructing framed sequences and applying a tapering window - -`tf.contrib.signal.overlap_and_add` can be used to reconstruct a signal from a -framed representation. For example, the following code reconstructs the signal -produced in the preceding example: - -```python -# Reconstructs `signals` from `frames` produced in the above example. However, -# the magnitude of `reconstructed_signals` will be greater than `signals`. -reconstructed_signals = tf.contrib.signal.overlap_and_add(frames, frame_step=32) -``` - -Note that because `frame_step` is 25% of `frame_length` in the above example, -the resulting reconstruction will have a greater magnitude than the original -`signals`. To compensate for this, we can use a tapering window function. If the -window function satisfies the Constant Overlap-Add (COLA) property for the given -frame step, then it will recover the original `signals`. - -`tf.contrib.signal.hamming_window` and `tf.contrib.signal.hann_window` both -satisfy the COLA property for a 75% overlap. - -```python -frame_length = 128 -frame_step = 32 -windowed_frames = frames * tf.contrib.signal.hann_window(frame_length) -reconstructed_signals = tf.contrib.signal.overlap_and_add( - windowed_frames, frame_step) -``` - -## Computing spectrograms - -A spectrogram is a time-frequency decomposition of a signal that indicates its -frequency content over time. The most common approach to computing spectrograms -is to take the magnitude of the [Short-time Fourier Transform][stft] (STFT), -which `tf.contrib.signal.stft` can compute as follows: - -```python -# A batch of float32 time-domain signals in the range [-1, 1] with shape -# [batch_size, signal_length]. Both batch_size and signal_length may be unknown. -signals = tf.placeholder(tf.float32, [None, None]) - -# `stfts` is a complex64 Tensor representing the Short-time Fourier Transform of -# each signal in `signals`. Its shape is [batch_size, ?, fft_unique_bins] -# where fft_unique_bins = fft_length // 2 + 1 = 513. -stfts = tf.contrib.signal.stft(signals, frame_length=1024, frame_step=512, - fft_length=1024) - -# A power spectrogram is the squared magnitude of the complex-valued STFT. -# A float32 Tensor of shape [batch_size, ?, 513]. -power_spectrograms = tf.real(stfts * tf.conj(stfts)) - -# An energy spectrogram is the magnitude of the complex-valued STFT. -# A float32 Tensor of shape [batch_size, ?, 513]. -magnitude_spectrograms = tf.abs(stfts) -``` - -You may use a power spectrogram or a magnitude spectrogram; each has its -advantages. Note that if you apply logarithmic compression, the power -spectrogram and magnitude spectrogram will differ by a factor of 2. - -## Logarithmic compression - -It is common practice to apply a compressive nonlinearity such as a logarithm or -power-law compression to spectrograms. This helps to balance the importance of -detail in low and high energy regions of the spectrum, which more closely -matches human auditory sensitivity. - -When compressing with a logarithm, it's a good idea to use a stabilizing offset -to avoid high dynamic ranges caused by the singularity at zero. - -```python -log_offset = 1e-6 -log_magnitude_spectrograms = tf.log(magnitude_spectrograms + log_offset) -``` - -## Computing log-mel spectrograms - -When working with spectral representations of audio, the [mel scale][mel] is a -common reweighting of the frequency dimension, which results in a -lower-dimensional and more perceptually-relevant representation of the audio. - -`tf.contrib.signal.linear_to_mel_weight_matrix` produces a matrix you can use -to convert a spectrogram to the mel scale. - -```python -# Warp the linear-scale, magnitude spectrograms into the mel-scale. -num_spectrogram_bins = magnitude_spectrograms.shape[-1].value -lower_edge_hertz, upper_edge_hertz, num_mel_bins = 80.0, 7600.0, 64 -linear_to_mel_weight_matrix = tf.contrib.signal.linear_to_mel_weight_matrix( - num_mel_bins, num_spectrogram_bins, sample_rate, lower_edge_hertz, - upper_edge_hertz) -mel_spectrograms = tf.tensordot( - magnitude_spectrograms, linear_to_mel_weight_matrix, 1) -# Note: Shape inference for `tf.tensordot` does not currently handle this case. -mel_spectrograms.set_shape(magnitude_spectrograms.shape[:-1].concatenate( - linear_to_mel_weight_matrix.shape[-1:])) -``` - -If desired, compress the mel spectrogram magnitudes. For example, you may use -logarithmic compression (as discussed in the previous section). - -Order matters! Compressing the spectrogram magnitudes after -reweighting the frequencies is different from reweighting the compressed -spectrogram magnitudes. According to the perceptual justification of the mel -scale, conversion from linear scale entails summing intensity or energy among -adjacent bands, i.e. it should be applied before logarithmic compression. Taking -the weighted sum of log-compressed values amounts to multiplying the -pre-logarithm values, which rarely, if ever, makes sense. - -```python -log_offset = 1e-6 -log_mel_spectrograms = tf.log(mel_spectrograms + log_offset) -``` - -## Computing Mel-Frequency Cepstral Coefficients (MFCCs) - -Call `tf.contrib.signal.mfccs_from_log_mel_spectrograms` to compute -[MFCCs][mfcc] from log-magnitude, mel-scale spectrograms (as computed in the -preceding example): - -```python -num_mfccs = 13 -# Keep the first `num_mfccs` MFCCs. -mfccs = tf.contrib.signal.mfccs_from_log_mel_spectrograms( - log_mel_spectrograms)[..., :num_mfccs] -``` - -[stft]: https://en.wikipedia.org/wiki/Short-time_Fourier_transform -[mel]: https://en.wikipedia.org/wiki/Mel_scale -[mfcc]: https://en.wikipedia.org/wiki/Mel-frequency_cepstrum diff --git a/tensorflow/docs_src/api_guides/python/contrib.staging.md b/tensorflow/docs_src/api_guides/python/contrib.staging.md deleted file mode 100644 index de143a7bd3..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.staging.md +++ /dev/null @@ -1,6 +0,0 @@ -# Staging (contrib) -[TOC] - -This library contains utilities for adding pipelining to a model. - -* `tf.contrib.staging.StagingArea` diff --git a/tensorflow/docs_src/api_guides/python/contrib.training.md b/tensorflow/docs_src/api_guides/python/contrib.training.md deleted file mode 100644 index 068efdc829..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.training.md +++ /dev/null @@ -1,50 +0,0 @@ -# Training (contrib) -[TOC] - -Training and input utilities. - -## Splitting sequence inputs into minibatches with state saving - -Use `tf.contrib.training.SequenceQueueingStateSaver` or -its wrapper `tf.contrib.training.batch_sequences_with_states` if -you have input data with a dynamic primary time / frame count axis which -you'd like to convert into fixed size segments during minibatching, and would -like to store state in the forward direction across segments of an example. - -* `tf.contrib.training.batch_sequences_with_states` -* `tf.contrib.training.NextQueuedSequenceBatch` -* `tf.contrib.training.SequenceQueueingStateSaver` - - -## Online data resampling - -To resample data with replacement on a per-example basis, use -`tf.contrib.training.rejection_sample` or -`tf.contrib.training.resample_at_rate`. For `rejection_sample`, provide -a boolean Tensor describing whether to accept or reject. Resulting batch sizes -are always the same. For `resample_at_rate`, provide the desired rate for each -example. Resulting batch sizes may vary. If you wish to specify relative -rates, rather than absolute ones, use `tf.contrib.training.weighted_resample` -(which also returns the actual resampling rate used for each output example). - -Use `tf.contrib.training.stratified_sample` to resample without replacement -from the data to achieve a desired mix of class proportions that the Tensorflow -graph sees. For instance, if you have a binary classification dataset that is -99.9% class 1, a common approach is to resample from the data so that the data -is more balanced. - -* `tf.contrib.training.rejection_sample` -* `tf.contrib.training.resample_at_rate` -* `tf.contrib.training.stratified_sample` -* `tf.contrib.training.weighted_resample` - -## Bucketing - -Use `tf.contrib.training.bucket` or -`tf.contrib.training.bucket_by_sequence_length` to stratify -minibatches into groups ("buckets"). Use `bucket_by_sequence_length` -with the argument `dynamic_pad=True` to receive minibatches of similarly -sized sequences for efficient training via `dynamic_rnn`. - -* `tf.contrib.training.bucket` -* `tf.contrib.training.bucket_by_sequence_length` diff --git a/tensorflow/docs_src/api_guides/python/contrib.util.md b/tensorflow/docs_src/api_guides/python/contrib.util.md deleted file mode 100644 index e5fd97e9f2..0000000000 --- a/tensorflow/docs_src/api_guides/python/contrib.util.md +++ /dev/null @@ -1,12 +0,0 @@ -# Utilities (contrib) -[TOC] - -Utilities for dealing with Tensors. - -## Miscellaneous Utility Functions - -* `tf.contrib.util.constant_value` -* `tf.contrib.util.make_tensor_proto` -* `tf.contrib.util.make_ndarray` -* `tf.contrib.util.ops_used_by_graph_def` -* `tf.contrib.util.stripped_op_list_for_graph` diff --git a/tensorflow/docs_src/api_guides/python/control_flow_ops.md b/tensorflow/docs_src/api_guides/python/control_flow_ops.md deleted file mode 100644 index 42c86d9978..0000000000 --- a/tensorflow/docs_src/api_guides/python/control_flow_ops.md +++ /dev/null @@ -1,57 +0,0 @@ -# Control Flow - -Note: Functions taking `Tensor` arguments can also take anything accepted by -`tf.convert_to_tensor`. - -[TOC] - -## Control Flow Operations - -TensorFlow provides several operations and classes that you can use to control -the execution of operations and add conditional dependencies to your graph. - -* `tf.identity` -* `tf.tuple` -* `tf.group` -* `tf.no_op` -* `tf.count_up_to` -* `tf.cond` -* `tf.case` -* `tf.while_loop` - -## Logical Operators - -TensorFlow provides several operations that you can use to add logical operators -to your graph. - -* `tf.logical_and` -* `tf.logical_not` -* `tf.logical_or` -* `tf.logical_xor` - -## Comparison Operators - -TensorFlow provides several operations that you can use to add comparison -operators to your graph. - -* `tf.equal` -* `tf.not_equal` -* `tf.less` -* `tf.less_equal` -* `tf.greater` -* `tf.greater_equal` -* `tf.where` - -## Debugging Operations - -TensorFlow provides several operations that you can use to validate values and -debug your graph. - -* `tf.is_finite` -* `tf.is_inf` -* `tf.is_nan` -* `tf.verify_tensor_all_finite` -* `tf.check_numerics` -* `tf.add_check_numerics_ops` -* `tf.Assert` -* `tf.Print` diff --git a/tensorflow/docs_src/api_guides/python/framework.md b/tensorflow/docs_src/api_guides/python/framework.md deleted file mode 100644 index 40a6c0783a..0000000000 --- a/tensorflow/docs_src/api_guides/python/framework.md +++ /dev/null @@ -1,51 +0,0 @@ -# Building Graphs -[TOC] - -Classes and functions for building TensorFlow graphs. - -## Core graph data structures - -* `tf.Graph` -* `tf.Operation` -* `tf.Tensor` - -## Tensor types - -* `tf.DType` -* `tf.as_dtype` - -## Utility functions - -* `tf.device` -* `tf.container` -* `tf.name_scope` -* `tf.control_dependencies` -* `tf.convert_to_tensor` -* `tf.convert_to_tensor_or_indexed_slices` -* `tf.convert_to_tensor_or_sparse_tensor` -* `tf.get_default_graph` -* `tf.reset_default_graph` -* `tf.import_graph_def` -* `tf.load_file_system_library` -* `tf.load_op_library` - -## Graph collections - -* `tf.add_to_collection` -* `tf.get_collection` -* `tf.get_collection_ref` -* `tf.GraphKeys` - -## Defining new operations - -* `tf.RegisterGradient` -* `tf.NotDifferentiable` -* `tf.NoGradient` -* `tf.TensorShape` -* `tf.Dimension` -* `tf.op_scope` -* `tf.get_seed` - -## For libraries building on TensorFlow - -* `tf.register_tensor_conversion_function` diff --git a/tensorflow/docs_src/api_guides/python/functional_ops.md b/tensorflow/docs_src/api_guides/python/functional_ops.md deleted file mode 100644 index 0a9fe02ad5..0000000000 --- a/tensorflow/docs_src/api_guides/python/functional_ops.md +++ /dev/null @@ -1,18 +0,0 @@ -# Higher Order Functions - -Note: Functions taking `Tensor` arguments can also take anything accepted by -`tf.convert_to_tensor`. - -[TOC] - -Functional operations. - -## Higher Order Operators - -TensorFlow provides several higher order operators to simplify the common -map-reduce programming patterns. - -* `tf.map_fn` -* `tf.foldl` -* `tf.foldr` -* `tf.scan` diff --git a/tensorflow/docs_src/api_guides/python/image.md b/tensorflow/docs_src/api_guides/python/image.md deleted file mode 100644 index c51b92db05..0000000000 --- a/tensorflow/docs_src/api_guides/python/image.md +++ /dev/null @@ -1,144 +0,0 @@ -# Images - -Note: Functions taking `Tensor` arguments can also take anything accepted by -`tf.convert_to_tensor`. - -[TOC] - -## Encoding and Decoding - -TensorFlow provides Ops to decode and encode JPEG and PNG formats. Encoded -images are represented by scalar string Tensors, decoded images by 3-D uint8 -tensors of shape `[height, width, channels]`. (PNG also supports uint16.) - -The encode and decode Ops apply to one image at a time. Their input and output -are all of variable size. If you need fixed size images, pass the output of -the decode Ops to one of the cropping and resizing Ops. - -Note: The PNG encode and decode Ops support RGBA, but the conversions Ops -presently only support RGB, HSV, and GrayScale. Presently, the alpha channel has -to be stripped from the image and re-attached using slicing ops. - -* `tf.image.decode_bmp` -* `tf.image.decode_gif` -* `tf.image.decode_jpeg` -* `tf.image.encode_jpeg` -* `tf.image.decode_png` -* `tf.image.encode_png` -* `tf.image.decode_image` - -## Resizing - -The resizing Ops accept input images as tensors of several types. They always -output resized images as float32 tensors. - -The convenience function `tf.image.resize_images` supports both 4-D -and 3-D tensors as input and output. 4-D tensors are for batches of images, -3-D tensors for individual images. - -Other resizing Ops only support 4-D batches of images as input: -`tf.image.resize_area`, `tf.image.resize_bicubic`, -`tf.image.resize_bilinear`, -`tf.image.resize_nearest_neighbor`. - -Example: - -```python -# Decode a JPG image and resize it to 299 by 299 using default method. -image = tf.image.decode_jpeg(...) -resized_image = tf.image.resize_images(image, [299, 299]) -``` - -* `tf.image.resize_images` -* `tf.image.resize_area` -* `tf.image.resize_bicubic` -* `tf.image.resize_bilinear` -* `tf.image.resize_nearest_neighbor` - -## Cropping - -* `tf.image.resize_image_with_crop_or_pad` -* `tf.image.central_crop` -* `tf.image.pad_to_bounding_box` -* `tf.image.crop_to_bounding_box` -* `tf.image.extract_glimpse` -* `tf.image.crop_and_resize` - -## Flipping, Rotating and Transposing - -* `tf.image.flip_up_down` -* `tf.image.random_flip_up_down` -* `tf.image.flip_left_right` -* `tf.image.random_flip_left_right` -* `tf.image.transpose_image` -* `tf.image.rot90` - -## Converting Between Colorspaces - -Image ops work either on individual images or on batches of images, depending on -the shape of their input Tensor. - -If 3-D, the shape is `[height, width, channels]`, and the Tensor represents one -image. If 4-D, the shape is `[batch_size, height, width, channels]`, and the -Tensor represents `batch_size` images. - -Currently, `channels` can usefully be 1, 2, 3, or 4. Single-channel images are -grayscale, images with 3 channels are encoded as either RGB or HSV. Images -with 2 or 4 channels include an alpha channel, which has to be stripped from the -image before passing the image to most image processing functions (and can be -re-attached later). - -Internally, images are either stored in as one `float32` per channel per pixel -(implicitly, values are assumed to lie in `[0,1)`) or one `uint8` per channel -per pixel (values are assumed to lie in `[0,255]`). - -TensorFlow can convert between images in RGB or HSV. The conversion functions -work only on float images, so you need to convert images in other formats using -`tf.image.convert_image_dtype`. - -Example: - -```python -# Decode an image and convert it to HSV. -rgb_image = tf.image.decode_png(..., channels=3) -rgb_image_float = tf.image.convert_image_dtype(rgb_image, tf.float32) -hsv_image = tf.image.rgb_to_hsv(rgb_image) -``` - -* `tf.image.rgb_to_grayscale` -* `tf.image.grayscale_to_rgb` -* `tf.image.hsv_to_rgb` -* `tf.image.rgb_to_hsv` -* `tf.image.convert_image_dtype` - -## Image Adjustments - -TensorFlow provides functions to adjust images in various ways: brightness, -contrast, hue, and saturation. Each adjustment can be done with predefined -parameters or with random parameters picked from predefined intervals. Random -adjustments are often useful to expand a training set and reduce overfitting. - -If several adjustments are chained it is advisable to minimize the number of -redundant conversions by first converting the images to the most natural data -type and representation (RGB or HSV). - -* `tf.image.adjust_brightness` -* `tf.image.random_brightness` -* `tf.image.adjust_contrast` -* `tf.image.random_contrast` -* `tf.image.adjust_hue` -* `tf.image.random_hue` -* `tf.image.adjust_gamma` -* `tf.image.adjust_saturation` -* `tf.image.random_saturation` -* `tf.image.per_image_standardization` - -## Working with Bounding Boxes - -* `tf.image.draw_bounding_boxes` -* `tf.image.non_max_suppression` -* `tf.image.sample_distorted_bounding_box` - -## Denoising - -* `tf.image.total_variation` diff --git a/tensorflow/docs_src/api_guides/python/index.md b/tensorflow/docs_src/api_guides/python/index.md deleted file mode 100644 index a791a1432a..0000000000 --- a/tensorflow/docs_src/api_guides/python/index.md +++ /dev/null @@ -1,52 +0,0 @@ -# Python API Guides - -* [Asserts and boolean checks](check_ops.md) -* [Building Graphs](framework.md) -* [Constants, Sequences, and Random Values](constant_op.md) -* [Control Flow](control_flow_ops.md) -* [Data IO (Python functions)](python_io.md) -* [Exporting and Importing a MetaGraph](meta_graph.md) -* [Higher Order Functions](functional_ops.md) -* [Histograms](histogram_ops.md) -* [Images](image.md) -* [Inputs and Readers](io_ops.md) -* [Math](math_ops.md) -* [Neural Network](nn.md) -* [Reading data](reading_data.md) -* [Running Graphs](client.md) -* [Sparse Tensors](sparse_ops.md) -* [Spectral Functions](spectral_ops.md) -* [Strings](string_ops.md) -* [Summary Operations](summary.md) -* [TensorFlow Debugger](tfdbg.md) -* [Tensor Handle Operations](session_ops.md) -* [Tensor Transformations](array_ops.md) -* [Testing](test.md) -* [Training](train.md) -* [Variables](state_ops.md) -* [Wraps python functions](script_ops.md) -* [BayesFlow Entropy (contrib)](contrib.bayesflow.entropy.md) -* [BayesFlow Monte Carlo (contrib)](contrib.bayesflow.monte_carlo.md) -* [BayesFlow Stochastic Graph (contrib)](contrib.bayesflow.stochastic_graph.md) -* [BayesFlow Stochastic Tensors (contrib)](contrib.bayesflow.stochastic_tensor.md) -* [BayesFlow Variational Inference (contrib)](contrib.bayesflow.variational_inference.md) -* [Copying Graph Elements (contrib)](contrib.copy_graph.md) -* [CRF (contrib)](contrib.crf.md) -* [FFmpeg (contrib)](contrib.ffmpeg.md) -* [Framework (contrib)](contrib.framework.md) -* [Graph Editor (contrib)](contrib.graph_editor.md) -* [Integrate (contrib)](contrib.integrate.md) -* [Layers (contrib)](contrib.layers.md) -* [Learn (contrib)](contrib.learn.md) -* [Linear Algebra (contrib)](contrib.linalg.md) -* [Losses (contrib)](contrib.losses.md) -* [Metrics (contrib)](contrib.metrics.md) -* [Optimization (contrib)](contrib.opt.md) -* [Random variable transformations (contrib)](contrib.distributions.bijectors.md) -* [RNN and Cells (contrib)](contrib.rnn.md) -* [Seq2seq Library (contrib)](contrib.seq2seq.md) -* [Signal Processing (contrib)](contrib.signal.md) -* [Staging (contrib)](contrib.staging.md) -* [Statistical Distributions (contrib)](contrib.distributions.md) -* [Training (contrib)](contrib.training.md) -* [Utilities (contrib)](contrib.util.md) diff --git a/tensorflow/docs_src/api_guides/python/input_dataset.md b/tensorflow/docs_src/api_guides/python/input_dataset.md deleted file mode 100644 index 911a76c2df..0000000000 --- a/tensorflow/docs_src/api_guides/python/input_dataset.md +++ /dev/null @@ -1,85 +0,0 @@ -# Dataset Input Pipeline -[TOC] - -`tf.data.Dataset` allows you to build complex input pipelines. See the -[Importing Data](../../guide/datasets.md) for an in-depth explanation of how to use this API. - -## Reader classes - -Classes that create a dataset from input files. - -* `tf.data.FixedLengthRecordDataset` -* `tf.data.TextLineDataset` -* `tf.data.TFRecordDataset` - -## Creating new datasets - -Static methods in `Dataset` that create new datasets. - -* `tf.data.Dataset.from_generator` -* `tf.data.Dataset.from_tensor_slices` -* `tf.data.Dataset.from_tensors` -* `tf.data.Dataset.list_files` -* `tf.data.Dataset.range` -* `tf.data.Dataset.zip` - -## Transformations on existing datasets - -These functions transform an existing dataset, and return a new dataset. Calls -can be chained together, as shown in the example below: - -``` -train_data = train_data.batch(100).shuffle().repeat() -``` - -* `tf.data.Dataset.apply` -* `tf.data.Dataset.batch` -* `tf.data.Dataset.cache` -* `tf.data.Dataset.concatenate` -* `tf.data.Dataset.filter` -* `tf.data.Dataset.flat_map` -* `tf.data.Dataset.interleave` -* `tf.data.Dataset.map` -* `tf.data.Dataset.padded_batch` -* `tf.data.Dataset.prefetch` -* `tf.data.Dataset.repeat` -* `tf.data.Dataset.shard` -* `tf.data.Dataset.shuffle` -* `tf.data.Dataset.skip` -* `tf.data.Dataset.take` - -### Custom transformation functions - -Custom transformation functions can be applied to a `Dataset` using `tf.data.Dataset.apply`. Below are custom transformation functions from `tf.contrib.data`: - -* `tf.contrib.data.batch_and_drop_remainder` -* `tf.contrib.data.dense_to_sparse_batch` -* `tf.contrib.data.enumerate_dataset` -* `tf.contrib.data.group_by_window` -* `tf.contrib.data.ignore_errors` -* `tf.contrib.data.map_and_batch` -* `tf.contrib.data.padded_batch_and_drop_remainder` -* `tf.contrib.data.parallel_interleave` -* `tf.contrib.data.rejection_resample` -* `tf.contrib.data.scan` -* `tf.contrib.data.shuffle_and_repeat` -* `tf.contrib.data.unbatch` - -## Iterating over datasets - -These functions make a `tf.data.Iterator` from a `Dataset`. - -* `tf.data.Dataset.make_initializable_iterator` -* `tf.data.Dataset.make_one_shot_iterator` - -The `Iterator` class also contains static methods that create a `tf.data.Iterator` that can be used with multiple `Dataset` objects. - -* `tf.data.Iterator.from_structure` -* `tf.data.Iterator.from_string_handle` - -## Extra functions from `tf.contrib.data` - -* `tf.contrib.data.get_single_element` -* `tf.contrib.data.make_saveable_from_iterator` -* `tf.contrib.data.read_batch_features` - diff --git a/tensorflow/docs_src/api_guides/python/io_ops.md b/tensorflow/docs_src/api_guides/python/io_ops.md deleted file mode 100644 index d7ce6fdfde..0000000000 --- a/tensorflow/docs_src/api_guides/python/io_ops.md +++ /dev/null @@ -1,130 +0,0 @@ -# Inputs and Readers - -Note: Functions taking `Tensor` arguments can also take anything accepted by -`tf.convert_to_tensor`. - -[TOC] - -## Placeholders - -TensorFlow provides a placeholder operation that must be fed with data -on execution. For more info, see the section on [Feeding data](../../api_guides/python/reading_data.md#Feeding). - -* `tf.placeholder` -* `tf.placeholder_with_default` - -For feeding `SparseTensor`s which are composite type, -there is a convenience function: - -* `tf.sparse_placeholder` - -## Readers - -TensorFlow provides a set of Reader classes for reading data formats. -For more information on inputs and readers, see [Reading data](../../api_guides/python/reading_data.md). - -* `tf.ReaderBase` -* `tf.TextLineReader` -* `tf.WholeFileReader` -* `tf.IdentityReader` -* `tf.TFRecordReader` -* `tf.FixedLengthRecordReader` - -## Converting - -TensorFlow provides several operations that you can use to convert various data -formats into tensors. - -* `tf.decode_csv` -* `tf.decode_raw` - -- - - - -### Example protocol buffer - -TensorFlow's [recommended format for training examples](../../api_guides/python/reading_data.md#standard_tensorflow_format) -is serialized `Example` protocol buffers, [described -here](https://www.tensorflow.org/code/tensorflow/core/example/example.proto). -They contain `Features`, [described -here](https://www.tensorflow.org/code/tensorflow/core/example/feature.proto). - -* `tf.VarLenFeature` -* `tf.FixedLenFeature` -* `tf.FixedLenSequenceFeature` -* `tf.SparseFeature` -* `tf.parse_example` -* `tf.parse_single_example` -* `tf.parse_tensor` -* `tf.decode_json_example` - -## Queues - -TensorFlow provides several implementations of 'Queues', which are -structures within the TensorFlow computation graph to stage pipelines -of tensors together. The following describe the basic Queue interface -and some implementations. To see an example use, see [Threading and Queues](../../api_guides/python/threading_and_queues.md). - -* `tf.QueueBase` -* `tf.FIFOQueue` -* `tf.PaddingFIFOQueue` -* `tf.RandomShuffleQueue` -* `tf.PriorityQueue` - -## Conditional Accumulators - -* `tf.ConditionalAccumulatorBase` -* `tf.ConditionalAccumulator` -* `tf.SparseConditionalAccumulator` - -## Dealing with the filesystem - -* `tf.matching_files` -* `tf.read_file` -* `tf.write_file` - -## Input pipeline - -TensorFlow functions for setting up an input-prefetching pipeline. -Please see the [reading data how-to](../../api_guides/python/reading_data.md) -for context. - -### Beginning of an input pipeline - -The "producer" functions add a queue to the graph and a corresponding -`QueueRunner` for running the subgraph that fills that queue. - -* `tf.train.match_filenames_once` -* `tf.train.limit_epochs` -* `tf.train.input_producer` -* `tf.train.range_input_producer` -* `tf.train.slice_input_producer` -* `tf.train.string_input_producer` - -### Batching at the end of an input pipeline - -These functions add a queue to the graph to assemble a batch of -examples, with possible shuffling. They also add a `QueueRunner` for -running the subgraph that fills that queue. - -Use `tf.train.batch` or `tf.train.batch_join` for batching -examples that have already been well shuffled. Use -`tf.train.shuffle_batch` or -`tf.train.shuffle_batch_join` for examples that would -benefit from additional shuffling. - -Use `tf.train.batch` or `tf.train.shuffle_batch` if you want a -single thread producing examples to batch, or if you have a -single subgraph producing examples but you want to run it in *N* threads -(where you increase *N* until it can keep the queue full). Use -`tf.train.batch_join` or `tf.train.shuffle_batch_join` -if you have *N* different subgraphs producing examples to batch and you -want them run by *N* threads. Use `maybe_*` to enqueue conditionally. - -* `tf.train.batch` -* `tf.train.maybe_batch` -* `tf.train.batch_join` -* `tf.train.maybe_batch_join` -* `tf.train.shuffle_batch` -* `tf.train.maybe_shuffle_batch` -* `tf.train.shuffle_batch_join` -* `tf.train.maybe_shuffle_batch_join` diff --git a/tensorflow/docs_src/api_guides/python/math_ops.md b/tensorflow/docs_src/api_guides/python/math_ops.md deleted file mode 100644 index 6ec18f48ef..0000000000 --- a/tensorflow/docs_src/api_guides/python/math_ops.md +++ /dev/null @@ -1,200 +0,0 @@ -# Math - -Note: Functions taking `Tensor` arguments can also take anything accepted by -`tf.convert_to_tensor`. - -[TOC] - -Note: Elementwise binary operations in TensorFlow follow [numpy-style -broadcasting](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html). - -## Arithmetic Operators - -TensorFlow provides several operations that you can use to add basic arithmetic -operators to your graph. - -* `tf.add` -* `tf.subtract` -* `tf.multiply` -* `tf.scalar_mul` -* `tf.div` -* `tf.divide` -* `tf.truediv` -* `tf.floordiv` -* `tf.realdiv` -* `tf.truncatediv` -* `tf.floor_div` -* `tf.div_no_nan` -* `tf.truncatemod` -* `tf.floormod` -* `tf.mod` -* `tf.cross` - -## Basic Math Functions - -TensorFlow provides several operations that you can use to add basic -mathematical functions to your graph. - -* `tf.add_n` -* `tf.abs` -* `tf.negative` -* `tf.sign` -* `tf.reciprocal` -* `tf.square` -* `tf.round` -* `tf.sqrt` -* `tf.rsqrt` -* `tf.pow` -* `tf.exp` -* `tf.expm1` -* `tf.log` -* `tf.log1p` -* `tf.ceil` -* `tf.floor` -* `tf.maximum` -* `tf.minimum` -* `tf.cos` -* `tf.sin` -* `tf.lbeta` -* `tf.tan` -* `tf.acos` -* `tf.asin` -* `tf.atan` -* `tf.cosh` -* `tf.sinh` -* `tf.asinh` -* `tf.acosh` -* `tf.atanh` -* `tf.lgamma` -* `tf.digamma` -* `tf.erf` -* `tf.erfc` -* `tf.squared_difference` -* `tf.igamma` -* `tf.igammac` -* `tf.zeta` -* `tf.polygamma` -* `tf.betainc` -* `tf.rint` - -## Matrix Math Functions - -TensorFlow provides several operations that you can use to add linear algebra -functions on matrices to your graph. - -* `tf.diag` -* `tf.diag_part` -* `tf.trace` -* `tf.transpose` -* `tf.eye` -* `tf.matrix_diag` -* `tf.matrix_diag_part` -* `tf.matrix_band_part` -* `tf.matrix_set_diag` -* `tf.matrix_transpose` -* `tf.matmul` -* `tf.norm` -* `tf.matrix_determinant` -* `tf.matrix_inverse` -* `tf.cholesky` -* `tf.cholesky_solve` -* `tf.matrix_solve` -* `tf.matrix_triangular_solve` -* `tf.matrix_solve_ls` -* `tf.qr` -* `tf.self_adjoint_eig` -* `tf.self_adjoint_eigvals` -* `tf.svd` - - -## Tensor Math Function - -TensorFlow provides operations that you can use to add tensor functions to your -graph. - -* `tf.tensordot` - - -## Complex Number Functions - -TensorFlow provides several operations that you can use to add complex number -functions to your graph. - -* `tf.complex` -* `tf.conj` -* `tf.imag` -* `tf.angle` -* `tf.real` - - -## Reduction - -TensorFlow provides several operations that you can use to perform -common math computations that reduce various dimensions of a tensor. - -* `tf.reduce_sum` -* `tf.reduce_prod` -* `tf.reduce_min` -* `tf.reduce_max` -* `tf.reduce_mean` -* `tf.reduce_all` -* `tf.reduce_any` -* `tf.reduce_logsumexp` -* `tf.count_nonzero` -* `tf.accumulate_n` -* `tf.einsum` - -## Scan - -TensorFlow provides several operations that you can use to perform scans -(running totals) across one axis of a tensor. - -* `tf.cumsum` -* `tf.cumprod` - -## Segmentation - -TensorFlow provides several operations that you can use to perform common -math computations on tensor segments. -Here a segmentation is a partitioning of a tensor along -the first dimension, i.e. it defines a mapping from the first dimension onto -`segment_ids`. The `segment_ids` tensor should be the size of -the first dimension, `d0`, with consecutive IDs in the range `0` to `k`, -where `k<d0`. -In particular, a segmentation of a matrix tensor is a mapping of rows to -segments. - -For example: - -```python -c = tf.constant([[1,2,3,4], [-1,-2,-3,-4], [5,6,7,8]]) -tf.segment_sum(c, tf.constant([0, 0, 1])) - ==> [[0 0 0 0] - [5 6 7 8]] -``` - -* `tf.segment_sum` -* `tf.segment_prod` -* `tf.segment_min` -* `tf.segment_max` -* `tf.segment_mean` -* `tf.unsorted_segment_sum` -* `tf.sparse_segment_sum` -* `tf.sparse_segment_mean` -* `tf.sparse_segment_sqrt_n` - - -## Sequence Comparison and Indexing - -TensorFlow provides several operations that you can use to add sequence -comparison and index extraction to your graph. You can use these operations to -determine sequence differences and determine the indexes of specific values in -a tensor. - -* `tf.argmin` -* `tf.argmax` -* `tf.setdiff1d` -* `tf.where` -* `tf.unique` -* `tf.edit_distance` -* `tf.invert_permutation` diff --git a/tensorflow/docs_src/api_guides/python/meta_graph.md b/tensorflow/docs_src/api_guides/python/meta_graph.md deleted file mode 100644 index 5e8a8b4d0f..0000000000 --- a/tensorflow/docs_src/api_guides/python/meta_graph.md +++ /dev/null @@ -1,277 +0,0 @@ -# Exporting and Importing a MetaGraph - -A [`MetaGraph`](https://www.tensorflow.org/code/tensorflow/core/protobuf/meta_graph.proto) contains both a TensorFlow GraphDef -as well as associated metadata necessary for running computation in a -graph when crossing a process boundary. It can also be used for long -term storage of graphs. The MetaGraph contains the information required -to continue training, perform evaluation, or run inference on a previously trained graph. - -The APIs for exporting and importing the complete model are in -the `tf.train.Saver` class: -`tf.train.export_meta_graph` -and -`tf.train.import_meta_graph`. - -## What's in a MetaGraph - -The information contained in a MetaGraph is expressed as a -[`MetaGraphDef`](https://www.tensorflow.org/code/tensorflow/core/protobuf/meta_graph.proto) -protocol buffer. It contains the following fields: - -* [`MetaInfoDef`](https://www.tensorflow.org/code/tensorflow/core/protobuf/meta_graph.proto) for meta information, such as version and other user information. -* [`GraphDef`](https://www.tensorflow.org/code/tensorflow/core/framework/graph.proto) for describing the graph. -* [`SaverDef`](https://www.tensorflow.org/code/tensorflow/core/protobuf/saver.proto) for the saver. -* [`CollectionDef`](https://www.tensorflow.org/code/tensorflow/core/protobuf/meta_graph.proto) -map that further describes additional components of the model such as -[`Variables`](../../api_guides/python/state_ops.md), -`tf.train.QueueRunner`, etc. - -In order for a Python object to be serialized -to and from `MetaGraphDef`, the Python class must implement `to_proto()` and -`from_proto()` methods, and register them with the system using -`register_proto_function`. For example: - - ```Python - def to_proto(self, export_scope=None): - - """Converts a `Variable` to a `VariableDef` protocol buffer. - - Args: - export_scope: Optional `string`. Name scope to remove. - - Returns: - A `VariableDef` protocol buffer, or `None` if the `Variable` is not - in the specified name scope. - """ - if (export_scope is None or - self._variable.name.startswith(export_scope)): - var_def = variable_pb2.VariableDef() - var_def.variable_name = ops.strip_name_scope( - self._variable.name, export_scope) - var_def.initializer_name = ops.strip_name_scope( - self.initializer.name, export_scope) - var_def.snapshot_name = ops.strip_name_scope( - self._snapshot.name, export_scope) - if self._save_slice_info: - var_def.save_slice_info_def.MergeFrom(self._save_slice_info.to_proto( - export_scope=export_scope)) - return var_def - else: - return None - - @staticmethod - def from_proto(variable_def, import_scope=None): - """Returns a `Variable` object created from `variable_def`.""" - return Variable(variable_def=variable_def, import_scope=import_scope) - - ops.register_proto_function(ops.GraphKeys.GLOBAL_VARIABLES, - proto_type=variable_pb2.VariableDef, - to_proto=Variable.to_proto, - from_proto=Variable.from_proto) - ``` - -## Exporting a Complete Model to MetaGraph - -The API for exporting a running model as a MetaGraph is `export_meta_graph()`. - - ```Python - def export_meta_graph(filename=None, collection_list=None, as_text=False): - """Writes `MetaGraphDef` to save_path/filename. - - Args: - filename: Optional meta_graph filename including the path. - collection_list: List of string keys to collect. - as_text: If `True`, writes the meta_graph as an ASCII proto. - - Returns: - A `MetaGraphDef` proto. - """ - ``` - - A `collection` can contain any Python objects that users would like to - be able to uniquely identify and easily retrieve. These objects can be - special operations in the graph, such as `train_op`, or hyper parameters, - such as "learning rate". Users can specify the list of collections - they would like to export. If no `collection_list` is specified, - all collections in the model will be exported. - - The API returns a serialized protocol buffer. If `filename` is - specified, the protocol buffer will also be written to a file. - - Here are some of the typical usage models: - - * Export the default running graph: - - ```Python - # Build the model - ... - with tf.Session() as sess: - # Use the model - ... - # Export the model to /tmp/my-model.meta. - meta_graph_def = tf.train.export_meta_graph(filename='/tmp/my-model.meta') - ``` - - * Export the default running graph and only a subset of the collections. - - ```Python - meta_graph_def = tf.train.export_meta_graph( - filename='/tmp/my-model.meta', - collection_list=["input_tensor", "output_tensor"]) - ``` - - -The MetaGraph is also automatically exported via the `save()` API in -`tf.train.Saver`. - - -## Import a MetaGraph - -The API for importing a MetaGraph file into a graph is `import_meta_graph()`. - -Here are some of the typical usage models: - -* Import and continue training without building the model from scratch. - - ```Python - ... - # Create a saver. - saver = tf.train.Saver(...variables...) - # Remember the training_op we want to run by adding it to a collection. - tf.add_to_collection('train_op', train_op) - sess = tf.Session() - for step in xrange(1000000): - sess.run(train_op) - if step % 1000 == 0: - # Saves checkpoint, which by default also exports a meta_graph - # named 'my-model-global_step.meta'. - saver.save(sess, 'my-model', global_step=step) - ``` - - Later we can continue training from this saved `meta_graph` without building - the model from scratch. - - ```Python - with tf.Session() as sess: - new_saver = tf.train.import_meta_graph('my-save-dir/my-model-10000.meta') - new_saver.restore(sess, 'my-save-dir/my-model-10000') - # tf.get_collection() returns a list. In this example we only want the - # first one. - train_op = tf.get_collection('train_op')[0] - for step in xrange(1000000): - sess.run(train_op) - ``` - -* Import and extend the graph. - - For example, we can first build an inference graph, export it as a meta graph: - - ```Python - # Creates an inference graph. - # Hidden 1 - images = tf.constant(1.2, tf.float32, shape=[100, 28]) - with tf.name_scope("hidden1"): - weights = tf.Variable( - tf.truncated_normal([28, 128], - stddev=1.0 / math.sqrt(float(28))), - name="weights") - biases = tf.Variable(tf.zeros([128]), - name="biases") - hidden1 = tf.nn.relu(tf.matmul(images, weights) + biases) - # Hidden 2 - with tf.name_scope("hidden2"): - weights = tf.Variable( - tf.truncated_normal([128, 32], - stddev=1.0 / math.sqrt(float(128))), - name="weights") - biases = tf.Variable(tf.zeros([32]), - name="biases") - hidden2 = tf.nn.relu(tf.matmul(hidden1, weights) + biases) - # Linear - with tf.name_scope("softmax_linear"): - weights = tf.Variable( - tf.truncated_normal([32, 10], - stddev=1.0 / math.sqrt(float(32))), - name="weights") - biases = tf.Variable(tf.zeros([10]), - name="biases") - logits = tf.matmul(hidden2, weights) + biases - tf.add_to_collection("logits", logits) - - init_all_op = tf.global_variables_initializer() - - with tf.Session() as sess: - # Initializes all the variables. - sess.run(init_all_op) - # Runs to logit. - sess.run(logits) - # Creates a saver. - saver0 = tf.train.Saver() - saver0.save(sess, 'my-save-dir/my-model-10000') - # Generates MetaGraphDef. - saver0.export_meta_graph('my-save-dir/my-model-10000.meta') - ``` - - Then later import it and extend it to a training graph. - - ```Python - with tf.Session() as sess: - new_saver = tf.train.import_meta_graph('my-save-dir/my-model-10000.meta') - new_saver.restore(sess, 'my-save-dir/my-model-10000') - # Addes loss and train. - labels = tf.constant(0, tf.int32, shape=[100], name="labels") - batch_size = tf.size(labels) - logits = tf.get_collection("logits")[0] - loss = tf.losses.sparse_softmax_cross_entropy(labels=labels, - logits=logits) - - tf.summary.scalar('loss', loss) - # Creates the gradient descent optimizer with the given learning rate. - optimizer = tf.train.GradientDescentOptimizer(0.01) - - # Runs train_op. - train_op = optimizer.minimize(loss) - sess.run(train_op) - ``` - -* Import a graph with preset devices. - - Sometimes an exported meta graph is from a training environment that the - importer doesn't have. For example, the model might have been trained - on GPUs, or in a distributed environment with replicas. When importing - such models, it's useful to be able to clear the device settings in - the graph so that we can run it on locally available devices. This can - be achieved by calling `import_meta_graph` with the `clear_devices` - option set to `True`. - - ```Python - with tf.Session() as sess: - new_saver = tf.train.import_meta_graph('my-save-dir/my-model-10000.meta', - clear_devices=True) - new_saver.restore(sess, 'my-save-dir/my-model-10000') - ... - ``` - -* Import within the default graph. - - Sometimes you might want to run `export_meta_graph` and `import_meta_graph` - in codelab using the default graph. In that case, you need to reset - the default graph by calling `tf.reset_default_graph()` first before - running import. - - ```Python - meta_graph_def = tf.train.export_meta_graph() - ... - tf.reset_default_graph() - ... - tf.train.import_meta_graph(meta_graph_def) - ... - ``` - -* Retrieve Hyper Parameters - - ```Python - filename = ".".join([tf.train.latest_checkpoint(train_dir), "meta"]) - tf.train.import_meta_graph(filename) - hparams = tf.get_collection("hparams") - ``` diff --git a/tensorflow/docs_src/api_guides/python/nn.md b/tensorflow/docs_src/api_guides/python/nn.md deleted file mode 100644 index 40dda3941d..0000000000 --- a/tensorflow/docs_src/api_guides/python/nn.md +++ /dev/null @@ -1,418 +0,0 @@ -# Neural Network - -Note: Functions taking `Tensor` arguments can also take anything accepted by -`tf.convert_to_tensor`. - -[TOC] - -## Activation Functions - -The activation ops provide different types of nonlinearities for use in neural -networks. These include smooth nonlinearities (`sigmoid`, `tanh`, `elu`, `selu`, -`softplus`, and `softsign`), continuous but not everywhere differentiable -functions (`relu`, `relu6`, `crelu` and `relu_x`), and random regularization -(`dropout`). - -All activation ops apply componentwise, and produce a tensor of the same -shape as the input tensor. - -* `tf.nn.relu` -* `tf.nn.relu6` -* `tf.nn.crelu` -* `tf.nn.elu` -* `tf.nn.selu` -* `tf.nn.softplus` -* `tf.nn.softsign` -* `tf.nn.dropout` -* `tf.nn.bias_add` -* `tf.sigmoid` -* `tf.tanh` - -## Convolution - -The convolution ops sweep a 2-D filter over a batch of images, applying the -filter to each window of each image of the appropriate size. The different -ops trade off between generic vs. specific filters: - -* `conv2d`: Arbitrary filters that can mix channels together. -* `depthwise_conv2d`: Filters that operate on each channel independently. -* `separable_conv2d`: A depthwise spatial filter followed by a pointwise filter. - -Note that although these ops are called "convolution", they are strictly -speaking "cross-correlation" since the filter is combined with an input window -without reversing the filter. For details, see [the properties of -cross-correlation](https://en.wikipedia.org/wiki/Cross-correlation#Properties). - -The filter is applied to image patches of the same size as the filter and -strided according to the `strides` argument. `strides = [1, 1, 1, 1]` applies -the filter to a patch at every offset, `strides = [1, 2, 2, 1]` applies the -filter to every other image patch in each dimension, etc. - -Ignoring channels for the moment, assume that the 4-D `input` has shape -`[batch, in_height, in_width, ...]` and the 4-D `filter` has shape -`[filter_height, filter_width, ...]`. The spatial semantics of the -convolution ops depend on the padding scheme chosen: `'SAME'` or `'VALID'`. -Note that the padding values are always zero. - -First, consider the `'SAME'` padding scheme. A detailed explanation of the -reasoning behind it is given in -[these notes](#Notes_on_SAME_Convolution_Padding). Here, we summarize the -mechanics of this padding scheme. When using `'SAME'`, the output height and -width are computed as: - - out_height = ceil(float(in_height) / float(strides[1])) - out_width = ceil(float(in_width) / float(strides[2])) - -The total padding applied along the height and width is computed as: - - if (in_height % strides[1] == 0): - pad_along_height = max(filter_height - strides[1], 0) - else: - pad_along_height = max(filter_height - (in_height % strides[1]), 0) - if (in_width % strides[2] == 0): - pad_along_width = max(filter_width - strides[2], 0) - else: - pad_along_width = max(filter_width - (in_width % strides[2]), 0) - -Finally, the padding on the top, bottom, left and right are: - - pad_top = pad_along_height // 2 - pad_bottom = pad_along_height - pad_top - pad_left = pad_along_width // 2 - pad_right = pad_along_width - pad_left - -Note that the division by 2 means that there might be cases when the padding on -both sides (top vs bottom, right vs left) are off by one. In this case, the -bottom and right sides always get the one additional padded pixel. For example, -when `pad_along_height` is 5, we pad 2 pixels at the top and 3 pixels at the -bottom. Note that this is different from existing libraries such as cuDNN and -Caffe, which explicitly specify the number of padded pixels and always pad the -same number of pixels on both sides. - -For the `'VALID'` scheme, the output height and width are computed as: - - out_height = ceil(float(in_height - filter_height + 1) / float(strides[1])) - out_width = ceil(float(in_width - filter_width + 1) / float(strides[2])) - -and no padding is used. - -Given the output size and the padding, the output can be computed as - -$$ output[b, i, j, :] = - sum_{d_i, d_j} input[b, strides[1] * i + d_i - pad_{top},\ - strides[2] * j + d_j - pad_{left}, ...] * - filter[d_i, d_j,\ ...]$$ - -where any value outside the original input image region are considered zero ( -i.e. we pad zero values around the border of the image). - -Since `input` is 4-D, each `input[b, i, j, :]` is a vector. For `conv2d`, these -vectors are multiplied by the `filter[di, dj, :, :]` matrices to produce new -vectors. For `depthwise_conv_2d`, each scalar component `input[b, i, j, k]` -is multiplied by a vector `filter[di, dj, k]`, and all the vectors are -concatenated. - -* `tf.nn.convolution` -* `tf.nn.conv2d` -* `tf.nn.depthwise_conv2d` -* `tf.nn.depthwise_conv2d_native` -* `tf.nn.separable_conv2d` -* `tf.nn.atrous_conv2d` -* `tf.nn.atrous_conv2d_transpose` -* `tf.nn.conv2d_transpose` -* `tf.nn.conv1d` -* `tf.nn.conv3d` -* `tf.nn.conv3d_transpose` -* `tf.nn.conv2d_backprop_filter` -* `tf.nn.conv2d_backprop_input` -* `tf.nn.conv3d_backprop_filter_v2` -* `tf.nn.depthwise_conv2d_native_backprop_filter` -* `tf.nn.depthwise_conv2d_native_backprop_input` - -## Pooling - -The pooling ops sweep a rectangular window over the input tensor, computing a -reduction operation for each window (average, max, or max with argmax). Each -pooling op uses rectangular windows of size `ksize` separated by offset -`strides`. For example, if `strides` is all ones every window is used, if -`strides` is all twos every other window is used in each dimension, etc. - -In detail, the output is - - output[i] = reduce(value[strides * i:strides * i + ksize]) - -where the indices also take into consideration the padding values. Please refer -to the `Convolution` section for details about the padding calculation. - -* `tf.nn.avg_pool` -* `tf.nn.max_pool` -* `tf.nn.max_pool_with_argmax` -* `tf.nn.avg_pool3d` -* `tf.nn.max_pool3d` -* `tf.nn.fractional_avg_pool` -* `tf.nn.fractional_max_pool` -* `tf.nn.pool` - -## Morphological filtering - -Morphological operators are non-linear filters used in image processing. - -[Greyscale morphological dilation -](https://en.wikipedia.org/wiki/Dilation_(morphology)) -is the max-sum counterpart of standard sum-product convolution: - -$$ output[b, y, x, c] = - max_{dy, dx} input[b, - strides[1] * y + rates[1] * dy, - strides[2] * x + rates[2] * dx, - c] + - filter[dy, dx, c]$$ - -The `filter` is usually called structuring function. Max-pooling is a special -case of greyscale morphological dilation when the filter assumes all-zero -values (a.k.a. flat structuring function). - -[Greyscale morphological erosion -](https://en.wikipedia.org/wiki/Erosion_(morphology)) -is the min-sum counterpart of standard sum-product convolution: - -$$ output[b, y, x, c] = - min_{dy, dx} input[b, - strides[1] * y - rates[1] * dy, - strides[2] * x - rates[2] * dx, - c] - - filter[dy, dx, c]$$ - -Dilation and erosion are dual to each other. The dilation of the input signal -`f` by the structuring signal `g` is equal to the negation of the erosion of -`-f` by the reflected `g`, and vice versa. - -Striding and padding is carried out in exactly the same way as in standard -convolution. Please refer to the `Convolution` section for details. - -* `tf.nn.dilation2d` -* `tf.nn.erosion2d` -* `tf.nn.with_space_to_batch` - -## Normalization - -Normalization is useful to prevent neurons from saturating when inputs may -have varying scale, and to aid generalization. - -* `tf.nn.l2_normalize` -* `tf.nn.local_response_normalization` -* `tf.nn.sufficient_statistics` -* `tf.nn.normalize_moments` -* `tf.nn.moments` -* `tf.nn.weighted_moments` -* `tf.nn.fused_batch_norm` -* `tf.nn.batch_normalization` -* `tf.nn.batch_norm_with_global_normalization` - -## Losses - -The loss ops measure error between two tensors, or between a tensor and zero. -These can be used for measuring accuracy of a network in a regression task -or for regularization purposes (weight decay). - -* `tf.nn.l2_loss` -* `tf.nn.log_poisson_loss` - -## Classification - -TensorFlow provides several operations that help you perform classification. - -* `tf.nn.sigmoid_cross_entropy_with_logits` -* `tf.nn.softmax` -* `tf.nn.log_softmax` -* `tf.nn.softmax_cross_entropy_with_logits` -* `tf.nn.softmax_cross_entropy_with_logits_v2` - identical to the base - version, except it allows gradient propagation into the labels. -* `tf.nn.sparse_softmax_cross_entropy_with_logits` -* `tf.nn.weighted_cross_entropy_with_logits` - -## Embeddings - -TensorFlow provides library support for looking up values in embedding -tensors. - -* `tf.nn.embedding_lookup` -* `tf.nn.embedding_lookup_sparse` - -## Recurrent Neural Networks - -TensorFlow provides a number of methods for constructing Recurrent -Neural Networks. Most accept an `RNNCell`-subclassed object -(see the documentation for `tf.contrib.rnn`). - -* `tf.nn.dynamic_rnn` -* `tf.nn.bidirectional_dynamic_rnn` -* `tf.nn.raw_rnn` - -## Connectionist Temporal Classification (CTC) - -* `tf.nn.ctc_loss` -* `tf.nn.ctc_greedy_decoder` -* `tf.nn.ctc_beam_search_decoder` - -## Evaluation - -The evaluation ops are useful for measuring the performance of a network. -They are typically used at evaluation time. - -* `tf.nn.top_k` -* `tf.nn.in_top_k` - -## Candidate Sampling - -Do you want to train a multiclass or multilabel model with thousands -or millions of output classes (for example, a language model with a -large vocabulary)? Training with a full Softmax is slow in this case, -since all of the classes are evaluated for every training example. -Candidate Sampling training algorithms can speed up your step times by -only considering a small randomly-chosen subset of contrastive classes -(called candidates) for each batch of training examples. - -See our -[Candidate Sampling Algorithms -Reference](https://www.tensorflow.org/extras/candidate_sampling.pdf) - -### Sampled Loss Functions - -TensorFlow provides the following sampled loss functions for faster training. - -* `tf.nn.nce_loss` -* `tf.nn.sampled_softmax_loss` - -### Candidate Samplers - -TensorFlow provides the following samplers for randomly sampling candidate -classes when using one of the sampled loss functions above. - -* `tf.nn.uniform_candidate_sampler` -* `tf.nn.log_uniform_candidate_sampler` -* `tf.nn.learned_unigram_candidate_sampler` -* `tf.nn.fixed_unigram_candidate_sampler` - -### Miscellaneous candidate sampling utilities - -* `tf.nn.compute_accidental_hits` - -### Quantization ops - -* `tf.nn.quantized_conv2d` -* `tf.nn.quantized_relu_x` -* `tf.nn.quantized_max_pool` -* `tf.nn.quantized_avg_pool` - -## Notes on SAME Convolution Padding - -In these notes, we provide more background on the use of the `'SAME'` padding -scheme for convolution operations. - -Tensorflow uses the smallest possible padding to achieve the desired output -size. To understand what is done, consider the \\(1\\)-dimensional case. Denote -\\(n_i\\) and \\(n_o\\) the input and output sizes, respectively, and denote the -kernel size \\(k\\) and stride \\(s\\). As discussed in the -[Convolution section](#Convolution), for `'SAME'`, -\\(n_o = \left \lceil{\frac{n_i}{s}}\right \rceil\\). - -To achieve a desired output size \\(n_o\\), we need to pad the input such that the -output size after a `'VALID'` convolution is \\(n_o\\). In other words, we need to -have padding \\(p_i\\) such that: - -\begin{equation} -\left \lceil{\frac{n_i + p_i - k + 1}{s}}\right \rceil = n_o -\label{eq:tf_pad_1} -\end{equation} - -What is the smallest \\(p_i\\) that we could possibly use? In general, \\(\left -\lceil{\frac{x}{a}}\right \rceil = b\\) (with \\(a > 0\\)) means that \\(b-1 < -\frac{x}{a} \leq b\\), and the smallest integer \\(x\\) we can choose to satisfy -this is \\(x = a\cdot (b-1) + 1\\). The same applies to our problem; we need -\\(p_i\\) such that: - -\begin{equation} -n_i + p_i - k + 1 = s\cdot (n_o - 1) + 1 -\label{eq:tf_pad_2} -\end{equation} - -which leads to: - -\begin{equation} -p_i = s\cdot (n_o - 1) + k - n_i -\label{eq:tf_pad_3} -\end{equation} - -Note that this might lead to negative \\(p_i\\), since in some cases we might -already have more input samples than we actually need. Thus, - -\begin{equation} -p_i = max(s\cdot (n_o - 1) + k - n_i, 0) -\label{eq:tf_pad_4} -\end{equation} - -Remember that, for `'SAME'` padding, -\\(n_o = \left \lceil{\frac{n_i}{s}}\right \rceil\\), as mentioned above. -We need to analyze in detail two cases: - -- \\(n_i \text{ mod } s = 0\\) - -In this simple case, \\(n_o = \frac{n_i}{s}\\), and the expression for \\(p_i\\) -becomes: - -\begin{equation} -p_i = max(k - s, 0) -\label{eq:tf_pad_5} -\end{equation} - -- \\(n_i \text{ mod } s \neq 0\\) - -This case is more involved to parse. First, we write: - -\begin{equation} -n_i = s\cdot\left \lceil{\frac{n_i}{s}}\right \rceil -- s \left(\left \lceil{\frac{n_i}{s}}\right \rceil - - \left \lfloor{\frac{n_i}{s}}\right \rfloor\right) -+ (n_i \text{ mod } s) -\label{eq:tf_pad_6} -\end{equation} - -For the case where \\((n_i \text{ mod } s) \neq 0\\), we have \\(\left -\lceil{\frac{n_i}{s}}\right \rceil -\left \lfloor{\frac{n_i}{s}}\right \rfloor = -1\\), leading to: - -\begin{equation} -n_i = s\cdot\left \lceil{\frac{n_i}{s}}\right \rceil -- s -+ (n_i \text{ mod } s) -\label{eq:tf_pad_7} -\end{equation} - -We can use this expression to substitute \\(n_o = \left -\lceil{\frac{n_i}{s}}\right \rceil\\) and get: - -$$\begin{align} -p_i &= max\left(s\cdot \left(\frac{n_i + s - (n_i \text{ mod } s)}{s} - - 1\right) + k - n_i, 0\right) \nonumber\\ -&= max(n_i + s - (n_i \text{ mod } s) - s + k - n_i,0) \nonumber \\ -&= max(k - (n_i \text{ mod } s),0) -\label{eq:tf_pad_8} -\end{align}$$ - -### Final expression - -Putting all together, the total padding used by tensorflow's convolution with -`'SAME'` mode is: - -$$\begin{align} -p_i = - \begin{cases} - max(k - s, 0), & \text{if $(n_i \text{ mod } s) = 0$} \\ - max(k - (n_i \text{ mod } s),0), & \text{if $(n_i \text{ mod } s) \neq 0$} - \end{cases} - \label{eq:tf_pad_9} -\end{align}$$ - -This expression is exactly equal to the ones presented for `pad_along_height` -and `pad_along_width` in the [Convolution section](#Convolution). diff --git a/tensorflow/docs_src/api_guides/python/python_io.md b/tensorflow/docs_src/api_guides/python/python_io.md deleted file mode 100644 index e7e82a8701..0000000000 --- a/tensorflow/docs_src/api_guides/python/python_io.md +++ /dev/null @@ -1,29 +0,0 @@ -# Data IO (Python functions) -[TOC] - -A TFRecords file represents a sequence of (binary) strings. The format is not -random access, so it is suitable for streaming large amounts of data but not -suitable if fast sharding or other non-sequential access is desired. - -* `tf.python_io.TFRecordWriter` -* `tf.python_io.tf_record_iterator` -* `tf.python_io.TFRecordCompressionType` -* `tf.python_io.TFRecordOptions` - -- - - - -## TFRecords Format Details - -A TFRecords file contains a sequence of strings with CRC32C (32-bit CRC using -the Castagnoli polynomial) hashes. Each record has the format - - uint64 length - uint32 masked_crc32_of_length - byte data[length] - uint32 masked_crc32_of_data - -and the records are concatenated together to produce the file. CRCs are -[described here](https://en.wikipedia.org/wiki/Cyclic_redundancy_check), and -the mask of a CRC is - - masked_crc = ((crc >> 15) | (crc << 17)) + 0xa282ead8ul diff --git a/tensorflow/docs_src/api_guides/python/reading_data.md b/tensorflow/docs_src/api_guides/python/reading_data.md deleted file mode 100644 index 9f555ee85d..0000000000 --- a/tensorflow/docs_src/api_guides/python/reading_data.md +++ /dev/null @@ -1,522 +0,0 @@ -# Reading data - -Note: The preferred way to feed data into a tensorflow program is using the -[`tf.data` API](../../guide/datasets.md). - -There are four methods of getting data into a TensorFlow program: - -* `tf.data` API: Easily construct a complex input pipeline. (preferred method) -* Feeding: Python code provides the data when running each step. -* `QueueRunner`: a queue-based input pipeline reads the data from files - at the beginning of a TensorFlow graph. -* Preloaded data: a constant or variable in the TensorFlow graph holds - all the data (for small data sets). - -[TOC] - -## `tf.data` API - -See the [Importing Data](../../guide/datasets.md) for an in-depth explanation of `tf.data.Dataset`. -The `tf.data` API enables you to extract and preprocess data -from different input/file formats, and apply transformations such as batching, -shuffling, and mapping functions over the dataset. This is an improved version -of the old input methods---feeding and `QueueRunner`---which are described -below for historical purposes. - -## Feeding - -Warning: "Feeding" is the least efficient way to feed data into a TensorFlow -program and should only be used for small experiments and debugging. - -TensorFlow's feed mechanism lets you inject data into any Tensor in a -computation graph. A Python computation can thus feed data directly into the -graph. - -Supply feed data through the `feed_dict` argument to a run() or eval() call -that initiates computation. - -```python -with tf.Session(): - input = tf.placeholder(tf.float32) - classifier = ... - print(classifier.eval(feed_dict={input: my_python_preprocessing_fn()})) -``` - -While you can replace any Tensor with feed data, including variables and -constants, the best practice is to use a -`tf.placeholder` node. A -`placeholder` exists solely to serve as the target of feeds. It is not -initialized and contains no data. A placeholder generates an error if -it is executed without a feed, so you won't forget to feed it. - -An example using `placeholder` and feeding to train on MNIST data can be found -in -[`tensorflow/examples/tutorials/mnist/fully_connected_feed.py`](https://www.tensorflow.org/code/tensorflow/examples/tutorials/mnist/fully_connected_feed.py). - -## `QueueRunner` - -Warning: This section discusses implementing input pipelines using the -queue-based APIs which can be cleanly replaced by the [`tf.data` -API](../../guide/datasets.md). - -A typical queue-based pipeline for reading records from files has the following stages: - -1. The list of filenames -2. *Optional* filename shuffling -3. *Optional* epoch limit -4. Filename queue -5. A Reader for the file format -6. A decoder for a record read by the reader -7. *Optional* preprocessing -8. Example queue - -### Filenames, shuffling, and epoch limits - -For the list of filenames, use either a constant string Tensor (like -`["file0", "file1"]` or `[("file%d" % i) for i in range(2)]`) or the -`tf.train.match_filenames_once` function. - -Pass the list of filenames to the `tf.train.string_input_producer` function. -`string_input_producer` creates a FIFO queue for holding the filenames until -the reader needs them. - -`string_input_producer` has options for shuffling and setting a maximum number -of epochs. A queue runner adds the whole list of filenames to the queue once -for each epoch, shuffling the filenames within an epoch if `shuffle=True`. -This procedure provides a uniform sampling of files, so that examples are not -under- or over- sampled relative to each other. - -The queue runner works in a thread separate from the reader that pulls -filenames from the queue, so the shuffling and enqueuing process does not -block the reader. - -### File formats - -Select the reader that matches your input file format and pass the filename -queue to the reader's read method. The read method outputs a key identifying -the file and record (useful for debugging if you have some weird records), and -a scalar string value. Use one (or more) of the decoder and conversion ops to -decode this string into the tensors that make up an example. - -#### CSV files - -To read text files in [comma-separated value (CSV) -format](https://tools.ietf.org/html/rfc4180), use a -`tf.TextLineReader` with the -`tf.decode_csv` operation. For example: - -```python -filename_queue = tf.train.string_input_producer(["file0.csv", "file1.csv"]) - -reader = tf.TextLineReader() -key, value = reader.read(filename_queue) - -# Default values, in case of empty columns. Also specifies the type of the -# decoded result. -record_defaults = [[1], [1], [1], [1], [1]] -col1, col2, col3, col4, col5 = tf.decode_csv( - value, record_defaults=record_defaults) -features = tf.stack([col1, col2, col3, col4]) - -with tf.Session() as sess: - # Start populating the filename queue. - coord = tf.train.Coordinator() - threads = tf.train.start_queue_runners(coord=coord) - - for i in range(1200): - # Retrieve a single instance: - example, label = sess.run([features, col5]) - - coord.request_stop() - coord.join(threads) -``` - -Each execution of `read` reads a single line from the file. The -`decode_csv` op then parses the result into a list of tensors. The -`record_defaults` argument determines the type of the resulting tensors and -sets the default value to use if a value is missing in the input string. - -You must call `tf.train.start_queue_runners` to populate the queue before -you call `run` or `eval` to execute the `read`. Otherwise `read` will -block while it waits for filenames from the queue. - -#### Fixed length records - -To read binary files in which each record is a fixed number of bytes, use -`tf.FixedLengthRecordReader` -with the `tf.decode_raw` operation. -The `decode_raw` op converts from a string to a uint8 tensor. - -For example, [the CIFAR-10 dataset](http://www.cs.toronto.edu/~kriz/cifar.html) -uses a file format where each record is represented using a fixed number of -bytes: 1 byte for the label followed by 3072 bytes of image data. Once you have -a uint8 tensor, standard operations can slice out each piece and reformat as -needed. For CIFAR-10, you can see how to do the reading and decoding in -[`tensorflow_models/tutorials/image/cifar10/cifar10_input.py`](https://github.com/tensorflow/models/tree/master/tutorials/image/cifar10/cifar10_input.py) -and described in -[this tutorial](../../tutorials/images/deep_cnn.md#prepare-the-data). - -#### Standard TensorFlow format - -Another approach is to convert whatever data you have into a supported format. -This approach makes it easier to mix and match data sets and network -architectures. The recommended format for TensorFlow is a -[TFRecords file](../../api_guides/python/python_io.md#tfrecords_format_details) -containing -[`tf.train.Example` protocol buffers](https://www.tensorflow.org/code/tensorflow/core/example/example.proto) -(which contain -[`Features`](https://www.tensorflow.org/code/tensorflow/core/example/feature.proto) -as a field). You write a little program that gets your data, stuffs it in an -`Example` protocol buffer, serializes the protocol buffer to a string, and then -writes the string to a TFRecords file using the -`tf.python_io.TFRecordWriter`. -For example, -[`tensorflow/examples/how_tos/reading_data/convert_to_records.py`](https://www.tensorflow.org/code/tensorflow/examples/how_tos/reading_data/convert_to_records.py) -converts MNIST data to this format. - -The recommended way to read a TFRecord file is with a `tf.data.TFRecordDataset`, [as in this example](https://www.tensorflow.org/code/tensorflow/examples/how_tos/reading_data/fully_connected_reader.py): - -``` python - dataset = tf.data.TFRecordDataset(filename) - dataset = dataset.repeat(num_epochs) - - # map takes a python function and applies it to every sample - dataset = dataset.map(decode) -``` - -To accomplish the same task with a queue based input pipeline requires the following code -(using the same `decode` function from the above example): - -``` python - filename_queue = tf.train.string_input_producer([filename], num_epochs=num_epochs) - reader = tf.TFRecordReader() - _, serialized_example = reader.read(filename_queue) - image,label = decode(serialized_example) -``` - -### Preprocessing - -You can then do any preprocessing of these examples you want. This would be any -processing that doesn't depend on trainable parameters. Examples include -normalization of your data, picking a random slice, adding noise or distortions, -etc. See -[`tensorflow_models/tutorials/image/cifar10/cifar10_input.py`](https://github.com/tensorflow/models/tree/master/tutorials/image/cifar10/cifar10_input.py) -for an example. - -### Batching - -At the end of the pipeline we use another queue to batch together examples for -training, evaluation, or inference. For this we use a queue that randomizes the -order of examples, using the -`tf.train.shuffle_batch`. - -Example: - -``` -def read_my_file_format(filename_queue): - reader = tf.SomeReader() - key, record_string = reader.read(filename_queue) - example, label = tf.some_decoder(record_string) - processed_example = some_processing(example) - return processed_example, label - -def input_pipeline(filenames, batch_size, num_epochs=None): - filename_queue = tf.train.string_input_producer( - filenames, num_epochs=num_epochs, shuffle=True) - example, label = read_my_file_format(filename_queue) - # min_after_dequeue defines how big a buffer we will randomly sample - # from -- bigger means better shuffling but slower start up and more - # memory used. - # capacity must be larger than min_after_dequeue and the amount larger - # determines the maximum we will prefetch. Recommendation: - # min_after_dequeue + (num_threads + a small safety margin) * batch_size - min_after_dequeue = 10000 - capacity = min_after_dequeue + 3 * batch_size - example_batch, label_batch = tf.train.shuffle_batch( - [example, label], batch_size=batch_size, capacity=capacity, - min_after_dequeue=min_after_dequeue) - return example_batch, label_batch -``` - -If you need more parallelism or shuffling of examples between files, use -multiple reader instances using the -`tf.train.shuffle_batch_join`. -For example: - -``` -def read_my_file_format(filename_queue): - # Same as above - -def input_pipeline(filenames, batch_size, read_threads, num_epochs=None): - filename_queue = tf.train.string_input_producer( - filenames, num_epochs=num_epochs, shuffle=True) - example_list = [read_my_file_format(filename_queue) - for _ in range(read_threads)] - min_after_dequeue = 10000 - capacity = min_after_dequeue + 3 * batch_size - example_batch, label_batch = tf.train.shuffle_batch_join( - example_list, batch_size=batch_size, capacity=capacity, - min_after_dequeue=min_after_dequeue) - return example_batch, label_batch -``` - -You still only use a single filename queue that is shared by all the readers. -That way we ensure that the different readers use different files from the same -epoch until all the files from the epoch have been started. (It is also usually -sufficient to have a single thread filling the filename queue.) - -An alternative is to use a single reader via the -`tf.train.shuffle_batch` -with `num_threads` bigger than 1. This will make it read from a single file at -the same time (but faster than with 1 thread), instead of N files at once. -This can be important: - -* If you have more reading threads than input files, to avoid the risk that - you will have two threads reading the same example from the same file near - each other. -* Or if reading N files in parallel causes too many disk seeks. - -How many threads do you need? the `tf.train.shuffle_batch*` functions add a -summary to the graph that indicates how full the example queue is. If you have -enough reading threads, that summary will stay above zero. You can -[view your summaries as training progresses using TensorBoard](../../guide/summaries_and_tensorboard.md). - -### Creating threads to prefetch using `QueueRunner` objects - -The short version: many of the `tf.train` functions listed above add -`tf.train.QueueRunner` objects to your -graph. These require that you call -`tf.train.start_queue_runners` -before running any training or inference steps, or it will hang forever. This -will start threads that run the input pipeline, filling the example queue so -that the dequeue to get the examples will succeed. This is best combined with a -`tf.train.Coordinator` to cleanly -shut down these threads when there are errors. If you set a limit on the number -of epochs, that will use an epoch counter that will need to be initialized. The -recommended code pattern combining these is: - -```python -# Create the graph, etc. -init_op = tf.global_variables_initializer() - -# Create a session for running operations in the Graph. -sess = tf.Session() - -# Initialize the variables (like the epoch counter). -sess.run(init_op) - -# Start input enqueue threads. -coord = tf.train.Coordinator() -threads = tf.train.start_queue_runners(sess=sess, coord=coord) - -try: - while not coord.should_stop(): - # Run training steps or whatever - sess.run(train_op) - -except tf.errors.OutOfRangeError: - print('Done training -- epoch limit reached') -finally: - # When done, ask the threads to stop. - coord.request_stop() - -# Wait for threads to finish. -coord.join(threads) -sess.close() -``` - -#### Aside: What is happening here? - -First we create the graph. It will have a few pipeline stages that are -connected by queues. The first stage will generate filenames to read and enqueue -them in the filename queue. The second stage consumes filenames (using a -`Reader`), produces examples, and enqueues them in an example queue. Depending -on how you have set things up, you may actually have a few independent copies of -the second stage, so that you can read from multiple files in parallel. At the -end of these stages is an enqueue operation, which enqueues into a queue that -the next stage dequeues from. We want to start threads running these enqueuing -operations, so that our training loop can dequeue examples from the example -queue. - -<div style="width:70%; margin-left:12%; margin-bottom:10px; margin-top:20px;"> -<img style="width:100%" src="https://www.tensorflow.org/images/AnimatedFileQueues.gif"> -</div> - -The helpers in `tf.train` that create these queues and enqueuing operations add -a `tf.train.QueueRunner` to the -graph using the -`tf.train.add_queue_runner` -function. Each `QueueRunner` is responsible for one stage, and holds the list of -enqueue operations that need to be run in threads. Once the graph is -constructed, the -`tf.train.start_queue_runners` -function asks each QueueRunner in the graph to start its threads running the -enqueuing operations. - -If all goes well, you can now run your training steps and the queues will be -filled by the background threads. If you have set an epoch limit, at some point -an attempt to dequeue examples will get an -`tf.errors.OutOfRangeError`. This -is the TensorFlow equivalent of "end of file" (EOF) -- this means the epoch -limit has been reached and no more examples are available. - -The last ingredient is the -`tf.train.Coordinator`. This is responsible -for letting all the threads know if anything has signaled a shut down. Most -commonly this would be because an exception was raised, for example one of the -threads got an error when running some operation (or an ordinary Python -exception). - -For more about threading, queues, QueueRunners, and Coordinators -[see here](../../api_guides/python/threading_and_queues.md). - -#### Aside: How clean shut-down when limiting epochs works - -Imagine you have a model that has set a limit on the number of epochs to train -on. That means that the thread generating filenames will only run that many -times before generating an `OutOfRange` error. The QueueRunner will catch that -error, close the filename queue, and exit the thread. Closing the queue does two -things: - -* Any future attempt to enqueue in the filename queue will generate an error. - At this point there shouldn't be any threads trying to do that, but this - is helpful when queues are closed due to other errors. -* Any current or future dequeue will either succeed (if there are enough - elements left) or fail (with an `OutOfRange` error) immediately. They won't - block waiting for more elements to be enqueued, since by the previous point - that can't happen. - -The point is that when the filename queue is closed, there will likely still be -many filenames in that queue, so the next stage of the pipeline (with the reader -and other preprocessing) may continue running for some time. Once the filename -queue is exhausted, though, the next attempt to dequeue a filename (e.g. from a -reader that has finished with the file it was working on) will trigger an -`OutOfRange` error. In this case, though, you might have multiple threads -associated with a single QueueRunner. If this isn't the last thread in the -QueueRunner, the `OutOfRange` error just causes the one thread to exit. This -allows the other threads, which are still finishing up their last file, to -proceed until they finish as well. (Assuming you are using a -`tf.train.Coordinator`, -other types of errors will cause all the threads to stop.) Once all the reader -threads hit the `OutOfRange` error, only then does the next queue, the example -queue, gets closed. - -Again, the example queue will have some elements queued, so training will -continue until those are exhausted. If the example queue is a -`tf.RandomShuffleQueue`, say -because you are using `shuffle_batch` or `shuffle_batch_join`, it normally will -avoid ever having fewer than its `min_after_dequeue` attr elements buffered. -However, once the queue is closed that restriction will be lifted and the queue -will eventually empty. At that point the actual training threads, when they -try and dequeue from example queue, will start getting `OutOfRange` errors and -exiting. Once all the training threads are done, -`tf.train.Coordinator.join` -will return and you can exit cleanly. - -### Filtering records or producing multiple examples per record - -Instead of examples with shapes `[x, y, z]`, you will produce a batch of -examples with shape `[batch, x, y, z]`. The batch size can be 0 if you want to -filter this record out (maybe it is in a hold-out set?), or bigger than 1 if you -are producing multiple examples per record. Then simply set `enqueue_many=True` -when calling one of the batching functions (such as `shuffle_batch` or -`shuffle_batch_join`). - -### Sparse input data - -SparseTensors don't play well with queues. If you use SparseTensors you have -to decode the string records using -`tf.parse_example` **after** -batching (instead of using `tf.parse_single_example` before batching). - -## Preloaded data - -This is only used for small data sets that can be loaded entirely in memory. -There are two approaches: - -* Store the data in a constant. -* Store the data in a variable, that you initialize (or assign to) and then - never change. - -Using a constant is a bit simpler, but uses more memory (since the constant is -stored inline in the graph data structure, which may be duplicated a few times). - -```python -training_data = ... -training_labels = ... -with tf.Session(): - input_data = tf.constant(training_data) - input_labels = tf.constant(training_labels) - ... -``` - -To instead use a variable, you need to also initialize it after the graph has been built. - -```python -training_data = ... -training_labels = ... -with tf.Session() as sess: - data_initializer = tf.placeholder(dtype=training_data.dtype, - shape=training_data.shape) - label_initializer = tf.placeholder(dtype=training_labels.dtype, - shape=training_labels.shape) - input_data = tf.Variable(data_initializer, trainable=False, collections=[]) - input_labels = tf.Variable(label_initializer, trainable=False, collections=[]) - ... - sess.run(input_data.initializer, - feed_dict={data_initializer: training_data}) - sess.run(input_labels.initializer, - feed_dict={label_initializer: training_labels}) -``` - -Setting `trainable=False` keeps the variable out of the -`GraphKeys.TRAINABLE_VARIABLES` collection in the graph, so we won't try and -update it when training. Setting `collections=[]` keeps the variable out of the -`GraphKeys.GLOBAL_VARIABLES` collection used for saving and restoring checkpoints. - -Either way, -`tf.train.slice_input_producer` -can be used to produce a slice at a time. This shuffles the examples across an -entire epoch, so further shuffling when batching is undesirable. So instead of -using the `shuffle_batch` functions, we use the plain -`tf.train.batch` function. To use -multiple preprocessing threads, set the `num_threads` parameter to a number -bigger than 1. - -An MNIST example that preloads the data using constants can be found in -[`tensorflow/examples/how_tos/reading_data/fully_connected_preloaded.py`](https://www.tensorflow.org/code/tensorflow/examples/how_tos/reading_data/fully_connected_preloaded.py), and one that preloads the data using variables can be found in -[`tensorflow/examples/how_tos/reading_data/fully_connected_preloaded_var.py`](https://www.tensorflow.org/code/tensorflow/examples/how_tos/reading_data/fully_connected_preloaded_var.py), -You can compare these with the `fully_connected_feed` and -`fully_connected_reader` versions above. - -## Multiple input pipelines - -Commonly you will want to train on one dataset and evaluate (or "eval") on -another. One way to do this is to actually have two separate graphs and -sessions, maybe in separate processes: - -* The training process reads training input data and periodically writes - checkpoint files with all the trained variables. -* The evaluation process restores the checkpoint files into an inference - model that reads validation input data. - -This is what is done `tf.estimator` and manually in -[the example CIFAR-10 model](../../tutorials/images/deep_cnn.md#save-and-restore-checkpoints). -This has a couple of benefits: - -* The eval is performed on a single snapshot of the trained variables. -* You can perform the eval even after training has completed and exited. - -You can have the train and eval in the same graph in the same process, and share -their trained variables or layers. See [the shared variables tutorial](../../guide/variables.md). - -To support the single-graph approach -[`tf.data`](../../guide/datasets.md) also supplies -[advanced iterator types](../../guide/datasets.md#creating_an_iterator) that -that allow the user to change the input pipeline without rebuilding the graph or -session. - -Note: Regardless of the implementation, many -operations (like `tf.layers.batch_normalization`, and `tf.layers.dropout`) -need to know if they are in training or evaluation mode, and you must be -careful to set this appropriately if you change the data source. diff --git a/tensorflow/docs_src/api_guides/python/regression_examples.md b/tensorflow/docs_src/api_guides/python/regression_examples.md deleted file mode 100644 index d67f38f57a..0000000000 --- a/tensorflow/docs_src/api_guides/python/regression_examples.md +++ /dev/null @@ -1,232 +0,0 @@ -# Regression Examples - -This unit provides the following short examples demonstrating how -to implement regression in Estimators: - -<table> - <tr> <th>Example</th> <th>Demonstrates How To...</th></tr> - - <tr> - <td><a href="https://www.tensorflow.org/code/tensorflow/examples/get_started/regression/linear_regression.py">linear_regression.py</a></td> - <td>Use the `tf.estimator.LinearRegressor` Estimator to train a - regression model on numeric data.</td> - </tr> - - <tr> - <td><a href="https://www.tensorflow.org/code/tensorflow/examples/get_started/regression/linear_regression_categorical.py">linear_regression_categorical.py</a></td> - <td>Use the `tf.estimator.LinearRegressor` Estimator to train a - regression model on categorical data.</td> - </tr> - - <tr> - <td><a href="https://www.tensorflow.org/code/tensorflow/examples/get_started/regression/dnn_regression.py">dnn_regression.py</a></td> - <td>Use the `tf.estimator.DNNRegressor` Estimator to train a - regression model on discrete data with a deep neural network.</td> - </tr> - - <tr> - <td><a href="https://www.tensorflow.org/code/tensorflow/examples/get_started/regression/custom_regression.py">custom_regression.py</a></td> - <td>Use `tf.estimator.Estimator` to train a customized dnn - regression model.</td> - </tr> - -</table> - -The preceding examples rely on the following data set utility: - -<table> - <tr> <th>Utility</th> <th>Description</th></tr> - - <tr> - <td><a href="https://www.tensorflow.org/code/tensorflow/examples/get_started/regression/imports85.py">imports85.py</a></td> - <td>This program provides utility functions that load the - <tt>imports85</tt> data set into formats that other TensorFlow - programs (for example, <tt>linear_regression.py</tt> and - <tt>dnn_regression.py</tt>) can use.</td> - </tr> - - -</table> - - -<!-- -## Linear regression concepts - -If you are new to machine learning and want to learn about regression, -watch the following video: - -(todo:jbgordon) Video introduction goes here. ---> - -<!-- -[When MLCC becomes available externally, add links to the relevant MLCC units.] ---> - - -<a name="running"></a> -## Running the examples - -You must [install TensorFlow](../../install/index.md) prior to running these examples. -Depending on the way you've installed TensorFlow, you might also -need to activate your TensorFlow environment. Then, do the following: - -1. Clone the TensorFlow repository from github. -2. `cd` to the top of the downloaded tree. -3. Check out the branch for you current tensorflow version: `git checkout rX.X` -4. `cd tensorflow/examples/get_started/regression`. - -You can now run any of the example TensorFlow programs in the -`tensorflow/examples/get_started/regression` directory as you -would run any Python program: - -```bsh -python linear_regressor.py -``` - -During training, all three programs output the following information: - -* The name of the checkpoint directory, which is important for TensorBoard. -* The training loss after every 100 iterations, which helps you - determine whether the model is converging. - -For example, here's some possible output for the `linear_regressor.py` -program: - -``` None -INFO:tensorflow:Saving checkpoints for 1 into /tmp/tmpAObiz9/model.ckpt. -INFO:tensorflow:loss = 161.308, step = 1 -INFO:tensorflow:global_step/sec: 1557.24 -INFO:tensorflow:loss = 15.7937, step = 101 (0.065 sec) -INFO:tensorflow:global_step/sec: 1529.17 -INFO:tensorflow:loss = 12.1988, step = 201 (0.065 sec) -INFO:tensorflow:global_step/sec: 1663.86 -... -INFO:tensorflow:loss = 6.99378, step = 901 (0.058 sec) -INFO:tensorflow:Saving checkpoints for 1000 into /tmp/tmpAObiz9/model.ckpt. -INFO:tensorflow:Loss for final step: 5.12413. -``` - - -<a name="basic"></a> -## linear_regressor.py - -`linear_regressor.py` trains a model that predicts the price of a car from -two numerical features. - -<table> - <tr> - <td>Estimator</td> - <td><tt>LinearRegressor</tt>, which is a pre-made Estimator for linear - regression.</td> - </tr> - - <tr> - <td>Features</td> - <td>Numerical: <tt>body-style</tt> and <tt>make</tt>.</td> - </tr> - - <tr> - <td>Label</td> - <td>Numerical: <tt>price</tt> - </tr> - - <tr> - <td>Algorithm</td> - <td>Linear regression.</td> - </tr> -</table> - -After training the model, the program concludes by outputting predicted -car prices for two car models. - - - -<a name="categorical"></a> -## linear_regression_categorical.py - -This program illustrates ways to represent categorical features. It -also demonstrates how to train a linear model based on a mix of -categorical and numerical features. - -<table> - <tr> - <td>Estimator</td> - <td><tt>LinearRegressor</tt>, which is a pre-made Estimator for linear - regression. </td> - </tr> - - <tr> - <td>Features</td> - <td>Categorical: <tt>curb-weight</tt> and <tt>highway-mpg</tt>.<br/> - Numerical: <tt>body-style</tt> and <tt>make</tt>.</td> - </tr> - - <tr> - <td>Label</td> - <td>Numerical: <tt>price</tt>.</td> - </tr> - - <tr> - <td>Algorithm</td> - <td>Linear regression.</td> - </tr> -</table> - - -<a name="dnn"></a> -## dnn_regression.py - -Like `linear_regression_categorical.py`, the `dnn_regression.py` example -trains a model that predicts the price of a car from two features. -Unlike `linear_regression_categorical.py`, the `dnn_regression.py` example uses -a deep neural network to train the model. Both examples rely on the same -features; `dnn_regression.py` demonstrates how to treat categorical features -in a deep neural network. - -<table> - <tr> - <td>Estimator</td> - <td><tt>DNNRegressor</tt>, which is a pre-made Estimator for - regression that relies on a deep neural network. The - `hidden_units` parameter defines the topography of the network.</td> - </tr> - - <tr> - <td>Features</td> - <td>Categorical: <tt>curb-weight</tt> and <tt>highway-mpg</tt>.<br/> - Numerical: <tt>body-style</tt> and <tt>make</tt>.</td> - </tr> - - <tr> - <td>Label</td> - <td>Numerical: <tt>price</tt>.</td> - </tr> - - <tr> - <td>Algorithm</td> - <td>Regression through a deep neural network.</td> - </tr> -</table> - -After printing loss values, the program outputs the Mean Square Error -on a test set. - - -<a name="dnn"></a> -## custom_regression.py - -The `custom_regression.py` example also trains a model that predicts the price -of a car based on mixed real-valued and categorical input features, described by -feature_columns. Unlike `linear_regression_categorical.py`, and -`dnn_regression.py` this example does not use a pre-made estimator, but defines -a custom model using the base `tf.estimator.Estimator` class. The -custom model is quite similar to the model defined by `dnn_regression.py`. - -The custom model is defined by the `model_fn` argument to the constructor. The -customization is made more reusable through `params` dictionary, which is later -passed through to the `model_fn` when the `model_fn` is called. - -The `model_fn` returns an -`tf.estimator.EstimatorSpec` which is a simple structure -indicating to the `Estimator` which operations should be run to accomplish -various tasks. diff --git a/tensorflow/docs_src/api_guides/python/session_ops.md b/tensorflow/docs_src/api_guides/python/session_ops.md deleted file mode 100644 index 5f41bcf209..0000000000 --- a/tensorflow/docs_src/api_guides/python/session_ops.md +++ /dev/null @@ -1,15 +0,0 @@ -# Tensor Handle Operations - -Note: Functions taking `Tensor` arguments can also take anything accepted by -`tf.convert_to_tensor`. - -[TOC] - -## Tensor Handle Operations - -TensorFlow provides several operators that allows the user to keep tensors -"in-place" across run calls. - -* `tf.get_session_handle` -* `tf.get_session_tensor` -* `tf.delete_session_tensor` diff --git a/tensorflow/docs_src/api_guides/python/sparse_ops.md b/tensorflow/docs_src/api_guides/python/sparse_ops.md deleted file mode 100644 index b360055ed0..0000000000 --- a/tensorflow/docs_src/api_guides/python/sparse_ops.md +++ /dev/null @@ -1,45 +0,0 @@ -# Sparse Tensors - -Note: Functions taking `Tensor` arguments can also take anything accepted by -`tf.convert_to_tensor`. - -[TOC] - -## Sparse Tensor Representation - -TensorFlow supports a `SparseTensor` representation for data that is sparse -in multiple dimensions. Contrast this representation with `IndexedSlices`, -which is efficient for representing tensors that are sparse in their first -dimension, and dense along all other dimensions. - -* `tf.SparseTensor` -* `tf.SparseTensorValue` - -## Conversion - -* `tf.sparse_to_dense` -* `tf.sparse_tensor_to_dense` -* `tf.sparse_to_indicator` -* `tf.sparse_merge` - -## Manipulation - -* `tf.sparse_concat` -* `tf.sparse_reorder` -* `tf.sparse_reshape` -* `tf.sparse_split` -* `tf.sparse_retain` -* `tf.sparse_reset_shape` -* `tf.sparse_fill_empty_rows` -* `tf.sparse_transpose` - -## Reduction -* `tf.sparse_reduce_sum` -* `tf.sparse_reduce_sum_sparse` - -## Math Operations -* `tf.sparse_add` -* `tf.sparse_softmax` -* `tf.sparse_tensor_dense_matmul` -* `tf.sparse_maximum` -* `tf.sparse_minimum` diff --git a/tensorflow/docs_src/api_guides/python/spectral_ops.md b/tensorflow/docs_src/api_guides/python/spectral_ops.md deleted file mode 100644 index f6d109a3a0..0000000000 --- a/tensorflow/docs_src/api_guides/python/spectral_ops.md +++ /dev/null @@ -1,26 +0,0 @@ -# Spectral Functions - -[TOC] - -The `tf.spectral` module supports several spectral decomposition operations -that you can use to transform Tensors of real and complex signals. - -## Discrete Fourier Transforms - -* `tf.spectral.fft` -* `tf.spectral.ifft` -* `tf.spectral.fft2d` -* `tf.spectral.ifft2d` -* `tf.spectral.fft3d` -* `tf.spectral.ifft3d` -* `tf.spectral.rfft` -* `tf.spectral.irfft` -* `tf.spectral.rfft2d` -* `tf.spectral.irfft2d` -* `tf.spectral.rfft3d` -* `tf.spectral.irfft3d` - -## Discrete Cosine Transforms - -* `tf.spectral.dct` -* `tf.spectral.idct` diff --git a/tensorflow/docs_src/api_guides/python/state_ops.md b/tensorflow/docs_src/api_guides/python/state_ops.md deleted file mode 100644 index fc55ea1481..0000000000 --- a/tensorflow/docs_src/api_guides/python/state_ops.md +++ /dev/null @@ -1,110 +0,0 @@ -# Variables - -Note: Functions taking `Tensor` arguments can also take anything accepted by -`tf.convert_to_tensor`. - -[TOC] - -## Variables - -* `tf.Variable` - -## Variable helper functions - -TensorFlow provides a set of functions to help manage the set of variables -collected in the graph. - -* `tf.global_variables` -* `tf.local_variables` -* `tf.model_variables` -* `tf.trainable_variables` -* `tf.moving_average_variables` -* `tf.global_variables_initializer` -* `tf.local_variables_initializer` -* `tf.variables_initializer` -* `tf.is_variable_initialized` -* `tf.report_uninitialized_variables` -* `tf.assert_variables_initialized` -* `tf.assign` -* `tf.assign_add` -* `tf.assign_sub` - -## Saving and Restoring Variables - -* `tf.train.Saver` -* `tf.train.latest_checkpoint` -* `tf.train.get_checkpoint_state` -* `tf.train.update_checkpoint_state` - -## Sharing Variables - -TensorFlow provides several classes and operations that you can use to -create variables contingent on certain conditions. - -* `tf.get_variable` -* `tf.get_local_variable` -* `tf.VariableScope` -* `tf.variable_scope` -* `tf.variable_op_scope` -* `tf.get_variable_scope` -* `tf.make_template` -* `tf.no_regularizer` -* `tf.constant_initializer` -* `tf.random_normal_initializer` -* `tf.truncated_normal_initializer` -* `tf.random_uniform_initializer` -* `tf.uniform_unit_scaling_initializer` -* `tf.zeros_initializer` -* `tf.ones_initializer` -* `tf.orthogonal_initializer` - -## Variable Partitioners for Sharding - -* `tf.fixed_size_partitioner` -* `tf.variable_axis_size_partitioner` -* `tf.min_max_variable_partitioner` - -## Sparse Variable Updates - -The sparse update ops modify a subset of the entries in a dense `Variable`, -either overwriting the entries or adding / subtracting a delta. These are -useful for training embedding models and similar lookup-based networks, since -only a small subset of embedding vectors change in any given step. - -Since a sparse update of a large tensor may be generated automatically during -gradient computation (as in the gradient of -`tf.gather`), -an `tf.IndexedSlices` class is provided that encapsulates a set -of sparse indices and values. `IndexedSlices` objects are detected and handled -automatically by the optimizers in most cases. - -* `tf.scatter_update` -* `tf.scatter_add` -* `tf.scatter_sub` -* `tf.scatter_mul` -* `tf.scatter_div` -* `tf.scatter_min` -* `tf.scatter_max` -* `tf.scatter_nd_update` -* `tf.scatter_nd_add` -* `tf.scatter_nd_sub` -* `tf.sparse_mask` -* `tf.IndexedSlices` - -### Read-only Lookup Tables - -* `tf.initialize_all_tables` -* `tf.tables_initializer` - - -## Exporting and Importing Meta Graphs - -* `tf.train.export_meta_graph` -* `tf.train.import_meta_graph` - -# Deprecated functions (removed after 2017-03-02). Please don't use them. - -* `tf.all_variables` -* `tf.initialize_all_variables` -* `tf.initialize_local_variables` -* `tf.initialize_variables` diff --git a/tensorflow/docs_src/api_guides/python/string_ops.md b/tensorflow/docs_src/api_guides/python/string_ops.md deleted file mode 100644 index 24a3aad642..0000000000 --- a/tensorflow/docs_src/api_guides/python/string_ops.md +++ /dev/null @@ -1,39 +0,0 @@ -# Strings - -Note: Functions taking `Tensor` arguments can also take anything accepted by -`tf.convert_to_tensor`. - -[TOC] - -## Hashing - -String hashing ops take a string input tensor and map each element to an -integer. - -* `tf.string_to_hash_bucket_fast` -* `tf.string_to_hash_bucket_strong` -* `tf.string_to_hash_bucket` - -## Joining - -String joining ops concatenate elements of input string tensors to produce a new -string tensor. - -* `tf.reduce_join` -* `tf.string_join` - -## Splitting - -* `tf.string_split` -* `tf.substr` - -## Conversion - -* `tf.as_string` -* `tf.string_to_number` - -* `tf.decode_raw` -* `tf.decode_csv` - -* `tf.encode_base64` -* `tf.decode_base64` diff --git a/tensorflow/docs_src/api_guides/python/summary.md b/tensorflow/docs_src/api_guides/python/summary.md deleted file mode 100644 index fc45e7b4c3..0000000000 --- a/tensorflow/docs_src/api_guides/python/summary.md +++ /dev/null @@ -1,23 +0,0 @@ -# Summary Operations -[TOC] - -Summaries provide a way to export condensed information about a model, which is -then accessible in tools such as [TensorBoard](../../guide/summaries_and_tensorboard.md). - -## Generation of Summaries - -### Class for writing Summaries -* `tf.summary.FileWriter` -* `tf.summary.FileWriterCache` - -### Summary Ops -* `tf.summary.tensor_summary` -* `tf.summary.scalar` -* `tf.summary.histogram` -* `tf.summary.audio` -* `tf.summary.image` -* `tf.summary.merge` -* `tf.summary.merge_all` - -## Utilities -* `tf.summary.get_summary_description` diff --git a/tensorflow/docs_src/api_guides/python/test.md b/tensorflow/docs_src/api_guides/python/test.md deleted file mode 100644 index b6e0a332b9..0000000000 --- a/tensorflow/docs_src/api_guides/python/test.md +++ /dev/null @@ -1,47 +0,0 @@ -# Testing -[TOC] - -## Unit tests - -TensorFlow provides a convenience class inheriting from `unittest.TestCase` -which adds methods relevant to TensorFlow tests. Here is an example: - -```python - import tensorflow as tf - - - class SquareTest(tf.test.TestCase): - - def testSquare(self): - with self.test_session(): - x = tf.square([2, 3]) - self.assertAllEqual(x.eval(), [4, 9]) - - - if __name__ == '__main__': - tf.test.main() -``` - -`tf.test.TestCase` inherits from `unittest.TestCase` but adds a few additional -methods. See `tf.test.TestCase` for details. - -* `tf.test.main` -* `tf.test.TestCase` -* `tf.test.test_src_dir_path` - -## Utilities - -Note: `tf.test.mock` is an alias to the python `mock` or `unittest.mock` -depending on the python version. - -* `tf.test.assert_equal_graph_def` -* `tf.test.get_temp_dir` -* `tf.test.is_built_with_cuda` -* `tf.test.is_gpu_available` -* `tf.test.gpu_device_name` - -## Gradient checking - -`tf.test.compute_gradient` and `tf.test.compute_gradient_error` perform -numerical differentiation of graphs for comparison against registered analytic -gradients. diff --git a/tensorflow/docs_src/api_guides/python/tfdbg.md b/tensorflow/docs_src/api_guides/python/tfdbg.md deleted file mode 100644 index 9778cdc0b0..0000000000 --- a/tensorflow/docs_src/api_guides/python/tfdbg.md +++ /dev/null @@ -1,50 +0,0 @@ -# TensorFlow Debugger -[TOC] - -Public Python API of TensorFlow Debugger (tfdbg). - -## Functions for adding debug watches - -These functions help you modify `RunOptions` to specify which `Tensor`s are to -be watched when the TensorFlow graph is executed at runtime. - -* `tfdbg.add_debug_tensor_watch` -* `tfdbg.watch_graph` -* `tfdbg.watch_graph_with_blacklists` - - -## Classes for debug-dump data and directories - -These classes allow you to load and inspect tensor values dumped from -TensorFlow graphs during runtime. - -* `tfdbg.DebugTensorDatum` -* `tfdbg.DebugDumpDir` - - -## Functions for loading debug-dump data - -* `tfdbg.load_tensor_from_event_file` - - -## Tensor-value predicates - -Built-in tensor-filter predicates to support conditional breakpoint between -runs. See `DebugDumpDir.find()` for more details. - -* `tfdbg.has_inf_or_nan` - - -## Session wrapper class and `SessionRunHook` implementations - -These classes allow you to - -* wrap aroundTensorFlow `Session` objects to debug plain TensorFlow models - (see `DumpingDebugWrapperSession` and `LocalCLIDebugWrapperSession`), or -* generate `SessionRunHook` objects to debug `tf.contrib.learn` models (see - `DumpingDebugHook` and `LocalCLIDebugHook`). - -* `tfdbg.DumpingDebugHook` -* `tfdbg.DumpingDebugWrapperSession` -* `tfdbg.LocalCLIDebugHook` -* `tfdbg.LocalCLIDebugWrapperSession` diff --git a/tensorflow/docs_src/api_guides/python/threading_and_queues.md b/tensorflow/docs_src/api_guides/python/threading_and_queues.md deleted file mode 100644 index e00f17f955..0000000000 --- a/tensorflow/docs_src/api_guides/python/threading_and_queues.md +++ /dev/null @@ -1,270 +0,0 @@ -# Threading and Queues - -Note: In versions of TensorFlow before 1.2, we recommended using multi-threaded, -queue-based input pipelines for performance. Beginning with TensorFlow 1.4, -however, we recommend using the `tf.data` module instead. (See -[Datasets](../../guide/datasets.md) for details. In TensorFlow 1.2 and 1.3, the module was -called `tf.contrib.data`.) The `tf.data` module offers an easier-to-use -interface for constructing efficient input pipelines. Furthermore, we've stopped -developing the old multi-threaded, queue-based input pipelines. We've retained -the documentation in this file to help developers who are still maintaining -older code. - -Multithreaded queues are a powerful and widely used mechanism supporting -asynchronous computation. - -Following the [dataflow programming model](graphs.md), TensorFlow's queues are -implemented using nodes in the computation graph. A queue is a stateful node, -like a variable: other nodes can modify its content. In particular, nodes can -enqueue new items in to the queue, or dequeue existing items from the -queue. TensorFlow's queues provide a way to coordinate multiple steps of a -computation: a queue will **block** any step that attempts to dequeue from it -when it is empty, or enqueue to it when it is full. When that condition no -longer holds, the queue will unblock the step and allow execution to proceed. - -TensorFlow implements several classes of queue. The principal difference between -these classes is the order that items are removed from the queue. To get a feel -for queues, let's consider a simple example. We will create a "first in, first -out" queue (`tf.FIFOQueue`) and fill it with zeros. Then we'll construct a -graph that takes an item off the queue, adds one to that item, and puts it back -on the end of the queue. Slowly, the numbers on the queue increase. - -<div style="width:70%; margin:auto; margin-bottom:10px; margin-top:20px;"> -<img style="width:100%" src="https://www.tensorflow.org/images/IncremeterFifoQueue.gif"> -</div> - -`Enqueue`, `EnqueueMany`, and `Dequeue` are special nodes. They take a pointer -to the queue instead of a normal value, allowing them to mutate its state. We -recommend that you think of these operations as being like methods of the queue -in an object-oriented sense. In fact, in the Python API, these operations are -created by calling methods on a queue object (e.g. `q.enqueue(...)`). - -Note: Queue methods (such as `q.enqueue(...)`) *must* run on the same device -as the queue. Incompatible device placement directives will be ignored when -creating these operations. - -Now that you have a bit of a feel for queues, let's dive into the details... - -## Queue usage overview - -Queues, such as `tf.FIFOQueue` -and `tf.RandomShuffleQueue`, -are important TensorFlow objects that aid in computing tensors asynchronously -in a graph. - -For example, a typical queue-based input pipeline uses a `RandomShuffleQueue` to -prepare inputs for training a model as follows: - -* Multiple threads prepare training examples and enqueue them. -* A training thread executes a training op that dequeues mini-batches from the - queue - -We recommend using the `tf.data.Dataset.shuffle` -and `tf.data.Dataset.batch` methods of a -`tf.data.Dataset` to accomplish this. However, if you'd prefer -to use a queue-based version instead, you can find a full implementation in the -`tf.train.shuffle_batch` function. - -For demonstration purposes a simplified implementation is given below. - -This function takes a source tensor, a capacity, and a batch size as arguments -and returns a tensor that dequeues a shuffled batch when executed. - -``` python -def simple_shuffle_batch(source, capacity, batch_size=10): - # Create a random shuffle queue. - queue = tf.RandomShuffleQueue(capacity=capacity, - min_after_dequeue=int(0.9*capacity), - shapes=source.shape, dtypes=source.dtype) - - # Create an op to enqueue one item. - enqueue = queue.enqueue(source) - - # Create a queue runner that, when started, will launch 4 threads applying - # that enqueue op. - num_threads = 4 - qr = tf.train.QueueRunner(queue, [enqueue] * num_threads) - - # Register the queue runner so it can be found and started by - # `tf.train.start_queue_runners` later (the threads are not launched yet). - tf.train.add_queue_runner(qr) - - # Create an op to dequeue a batch - return queue.dequeue_many(batch_size) -``` - -Once started by `tf.train.start_queue_runners`, or indirectly through -`tf.train.MonitoredSession`, the `QueueRunner` will launch the -threads in the background to fill the queue. Meanwhile the main thread will -execute the `dequeue_many` op to pull data from it. Note how these ops do not -depend on each other, except indirectly through the internal state of the queue. - -The simplest possible use of this function might be something like this: - -``` python -# create a dataset that counts from 0 to 99 -input = tf.constant(list(range(100))) -input = tf.data.Dataset.from_tensor_slices(input) -input = input.make_one_shot_iterator().get_next() - -# Create a slightly shuffled batch from the sorted elements -get_batch = simple_shuffle_batch(input, capacity=20) - -# `MonitoredSession` will start and manage the `QueueRunner` threads. -with tf.train.MonitoredSession() as sess: - # Since the `QueueRunners` have been started, data is available in the - # queue, so the `sess.run(get_batch)` call will not hang. - while not sess.should_stop(): - print(sess.run(get_batch)) -``` - -``` -[ 8 10 7 5 4 13 15 14 25 0] -[23 29 28 31 33 18 19 11 34 27] -[12 21 37 39 35 22 44 36 20 46] -... -``` - -For most use cases, the automatic thread startup and management provided -by `tf.train.MonitoredSession` is sufficient. In the rare case that it is not, -TensorFlow provides tools for manually managing your threads and queues. - -## Manual Thread Management - -As we have seen, the TensorFlow `Session` object is multithreaded and -thread-safe, so multiple threads can -easily use the same session and run ops in parallel. However, it is not always -easy to implement a Python program that drives threads as required. All -threads must be able to stop together, exceptions must be caught and -reported, and queues must be properly closed when stopping. - -TensorFlow provides two classes to help: -`tf.train.Coordinator` and -`tf.train.QueueRunner`. These two classes -are designed to be used together. The `Coordinator` class helps multiple threads -stop together and report exceptions to a program that waits for them to stop. -The `QueueRunner` class is used to create a number of threads cooperating to -enqueue tensors in the same queue. - -### Coordinator - -The `tf.train.Coordinator` class manages background threads in a TensorFlow -program and helps multiple threads stop together. - -Its key methods are: - -* `tf.train.Coordinator.should_stop`: returns `True` if the threads should stop. -* `tf.train.Coordinator.request_stop`: requests that threads should stop. -* `tf.train.Coordinator.join`: waits until the specified threads have stopped. - -You first create a `Coordinator` object, and then create a number of threads -that use the coordinator. The threads typically run loops that stop when -`should_stop()` returns `True`. - -Any thread can decide that the computation should stop. It only has to call -`request_stop()` and the other threads will stop as `should_stop()` will then -return `True`. - -```python -# Using Python's threading library. -import threading - -# Thread body: loop until the coordinator indicates a stop was requested. -# If some condition becomes true, ask the coordinator to stop. -def MyLoop(coord): - while not coord.should_stop(): - ...do something... - if ...some condition...: - coord.request_stop() - -# Main thread: create a coordinator. -coord = tf.train.Coordinator() - -# Create 10 threads that run 'MyLoop()' -threads = [threading.Thread(target=MyLoop, args=(coord,)) for i in xrange(10)] - -# Start the threads and wait for all of them to stop. -for t in threads: - t.start() -coord.join(threads) -``` - -Obviously, the coordinator can manage threads doing very different things. -They don't have to be all the same as in the example above. The coordinator -also has support to capture and report exceptions. See the `tf.train.Coordinator` documentation for more details. - -### QueueRunner - -The `tf.train.QueueRunner` class creates a number of threads that repeatedly -run an enqueue op. These threads can use a coordinator to stop together. In -addition, a queue runner will run a *closer operation* that closes the queue if -an exception is reported to the coordinator. - -You can use a queue runner to implement the architecture described above. - -First build a graph that uses a TensorFlow queue (e.g. a `tf.RandomShuffleQueue`) for input examples. Add ops that -process examples and enqueue them in the queue. Add training ops that start by -dequeueing from the queue. - -```python -example = ...ops to create one example... -# Create a queue, and an op that enqueues examples one at a time in the queue. -queue = tf.RandomShuffleQueue(...) -enqueue_op = queue.enqueue(example) -# Create a training graph that starts by dequeueing a batch of examples. -inputs = queue.dequeue_many(batch_size) -train_op = ...use 'inputs' to build the training part of the graph... -``` - -In the Python training program, create a `QueueRunner` that will run a few -threads to process and enqueue examples. Create a `Coordinator` and ask the -queue runner to start its threads with the coordinator. Write a training loop -that also uses the coordinator. - -```python -# Create a queue runner that will run 4 threads in parallel to enqueue -# examples. -qr = tf.train.QueueRunner(queue, [enqueue_op] * 4) - -# Launch the graph. -sess = tf.Session() -# Create a coordinator, launch the queue runner threads. -coord = tf.train.Coordinator() -enqueue_threads = qr.create_threads(sess, coord=coord, start=True) -# Run the training loop, controlling termination with the coordinator. -for step in xrange(1000000): - if coord.should_stop(): - break - sess.run(train_op) -# When done, ask the threads to stop. -coord.request_stop() -# And wait for them to actually do it. -coord.join(enqueue_threads) -``` - -### Handling exceptions - -Threads started by queue runners do more than just run the enqueue ops. They -also catch and handle exceptions generated by queues, including the -`tf.errors.OutOfRangeError` exception, which is used to report that a queue was -closed. - -A training program that uses a coordinator must similarly catch and report -exceptions in its main loop. - -Here is an improved version of the training loop above. - -```python -try: - for step in xrange(1000000): - if coord.should_stop(): - break - sess.run(train_op) -except Exception, e: - # Report exceptions to the coordinator. - coord.request_stop(e) -finally: - # Terminate as usual. It is safe to call `coord.request_stop()` twice. - coord.request_stop() - coord.join(threads) -``` diff --git a/tensorflow/docs_src/api_guides/python/train.md b/tensorflow/docs_src/api_guides/python/train.md deleted file mode 100644 index 4b4c6a4fe3..0000000000 --- a/tensorflow/docs_src/api_guides/python/train.md +++ /dev/null @@ -1,139 +0,0 @@ -# Training -[TOC] - -`tf.train` provides a set of classes and functions that help train models. - -## Optimizers - -The Optimizer base class provides methods to compute gradients for a loss and -apply gradients to variables. A collection of subclasses implement classic -optimization algorithms such as GradientDescent and Adagrad. - -You never instantiate the Optimizer class itself, but instead instantiate one -of the subclasses. - -* `tf.train.Optimizer` -* `tf.train.GradientDescentOptimizer` -* `tf.train.AdadeltaOptimizer` -* `tf.train.AdagradOptimizer` -* `tf.train.AdagradDAOptimizer` -* `tf.train.MomentumOptimizer` -* `tf.train.AdamOptimizer` -* `tf.train.FtrlOptimizer` -* `tf.train.ProximalGradientDescentOptimizer` -* `tf.train.ProximalAdagradOptimizer` -* `tf.train.RMSPropOptimizer` - -See `tf.contrib.opt` for more optimizers. - -## Gradient Computation - -TensorFlow provides functions to compute the derivatives for a given -TensorFlow computation graph, adding operations to the graph. The -optimizer classes automatically compute derivatives on your graph, but -creators of new Optimizers or expert users can call the lower-level -functions below. - -* `tf.gradients` -* `tf.AggregationMethod` -* `tf.stop_gradient` -* `tf.hessians` - - -## Gradient Clipping - -TensorFlow provides several operations that you can use to add clipping -functions to your graph. You can use these functions to perform general data -clipping, but they're particularly useful for handling exploding or vanishing -gradients. - -* `tf.clip_by_value` -* `tf.clip_by_norm` -* `tf.clip_by_average_norm` -* `tf.clip_by_global_norm` -* `tf.global_norm` - -## Decaying the learning rate - -* `tf.train.exponential_decay` -* `tf.train.inverse_time_decay` -* `tf.train.natural_exp_decay` -* `tf.train.piecewise_constant` -* `tf.train.polynomial_decay` -* `tf.train.cosine_decay` -* `tf.train.linear_cosine_decay` -* `tf.train.noisy_linear_cosine_decay` - -## Moving Averages - -Some training algorithms, such as GradientDescent and Momentum often benefit -from maintaining a moving average of variables during optimization. Using the -moving averages for evaluations often improve results significantly. - -* `tf.train.ExponentialMovingAverage` - -## Coordinator and QueueRunner - -See [Threading and Queues](../../api_guides/python/threading_and_queues.md) -for how to use threads and queues. For documentation on the Queue API, -see [Queues](../../api_guides/python/io_ops.md#queues). - - -* `tf.train.Coordinator` -* `tf.train.QueueRunner` -* `tf.train.LooperThread` -* `tf.train.add_queue_runner` -* `tf.train.start_queue_runners` - -## Distributed execution - -See [Distributed TensorFlow](../../deploy/distributed.md) for -more information about how to configure a distributed TensorFlow program. - -* `tf.train.Server` -* `tf.train.Supervisor` -* `tf.train.SessionManager` -* `tf.train.ClusterSpec` -* `tf.train.replica_device_setter` -* `tf.train.MonitoredTrainingSession` -* `tf.train.MonitoredSession` -* `tf.train.SingularMonitoredSession` -* `tf.train.Scaffold` -* `tf.train.SessionCreator` -* `tf.train.ChiefSessionCreator` -* `tf.train.WorkerSessionCreator` - -## Reading Summaries from Event Files - -See [Summaries and TensorBoard](../../guide/summaries_and_tensorboard.md) for an -overview of summaries, event files, and visualization in TensorBoard. - -* `tf.train.summary_iterator` - -## Training Hooks - -Hooks are tools that run in the process of training/evaluation of the model. - -* `tf.train.SessionRunHook` -* `tf.train.SessionRunArgs` -* `tf.train.SessionRunContext` -* `tf.train.SessionRunValues` -* `tf.train.LoggingTensorHook` -* `tf.train.StopAtStepHook` -* `tf.train.CheckpointSaverHook` -* `tf.train.NewCheckpointReader` -* `tf.train.StepCounterHook` -* `tf.train.NanLossDuringTrainingError` -* `tf.train.NanTensorHook` -* `tf.train.SummarySaverHook` -* `tf.train.GlobalStepWaiterHook` -* `tf.train.FinalOpsHook` -* `tf.train.FeedFnHook` - -## Training Utilities - -* `tf.train.global_step` -* `tf.train.basic_train_loop` -* `tf.train.get_global_step` -* `tf.train.assert_global_step` -* `tf.train.write_graph` |