blob: 94cf0de32a2d2ea16d1581e7c42a08b59aa52888 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
|
# Inputs and Readers
Note: Functions taking `Tensor` arguments can also take anything accepted by
@{tf.convert_to_tensor}.
[TOC]
## Placeholders
TensorFlow provides a placeholder operation that must be fed with data
on execution. For more info, see the section on @{$reading_data#feeding$Feeding data}.
* @{tf.placeholder}
* @{tf.placeholder_with_default}
For feeding `SparseTensor`s which are composite type,
there is a convenience function:
* @{tf.sparse_placeholder}
## Readers
TensorFlow provides a set of Reader classes for reading data formats.
For more information on inputs and readers, see @{$reading_data$Reading data}.
* @{tf.ReaderBase}
* @{tf.TextLineReader}
* @{tf.WholeFileReader}
* @{tf.IdentityReader}
* @{tf.TFRecordReader}
* @{tf.FixedLengthRecordReader}
## Converting
TensorFlow provides several operations that you can use to convert various data
formats into tensors.
* @{tf.decode_csv}
* @{tf.decode_raw}
- - -
### Example protocol buffer
TensorFlow's @{$reading_data#standard-tensorflow-format$recommended format for training examples}
is serialized `Example` protocol buffers, [described
here](https://www.tensorflow.org/code/tensorflow/core/example/example.proto).
They contain `Features`, [described
here](https://www.tensorflow.org/code/tensorflow/core/example/feature.proto).
* @{tf.VarLenFeature}
* @{tf.FixedLenFeature}
* @{tf.FixedLenSequenceFeature}
* @{tf.SparseFeature}
* @{tf.parse_example}
* @{tf.parse_single_example}
* @{tf.parse_tensor}
* @{tf.decode_json_example}
## Queues
TensorFlow provides several implementations of 'Queues', which are
structures within the TensorFlow computation graph to stage pipelines
of tensors together. The following describe the basic Queue interface
and some implementations. To see an example use, see @{$threading_and_queues$Threading and Queues}.
* @{tf.QueueBase}
* @{tf.FIFOQueue}
* @{tf.PaddingFIFOQueue}
* @{tf.RandomShuffleQueue}
* @{tf.PriorityQueue}
## Conditional Accumulators
* @{tf.ConditionalAccumulatorBase}
* @{tf.ConditionalAccumulator}
* @{tf.SparseConditionalAccumulator}
## Dealing with the filesystem
* @{tf.matching_files}
* @{tf.read_file}
* @{tf.write_file}
## Input pipeline
TensorFlow functions for setting up an input-prefetching pipeline.
Please see the @{$reading_data$reading data how-to}
for context.
### Beginning of an input pipeline
The "producer" functions add a queue to the graph and a corresponding
`QueueRunner` for running the subgraph that fills that queue.
* @{tf.train.match_filenames_once}
* @{tf.train.limit_epochs}
* @{tf.train.input_producer}
* @{tf.train.range_input_producer}
* @{tf.train.slice_input_producer}
* @{tf.train.string_input_producer}
### Batching at the end of an input pipeline
These functions add a queue to the graph to assemble a batch of
examples, with possible shuffling. They also add a `QueueRunner` for
running the subgraph that fills that queue.
Use @{tf.train.batch} or @{tf.train.batch_join} for batching
examples that have already been well shuffled. Use
@{tf.train.shuffle_batch} or
@{tf.train.shuffle_batch_join} for examples that would
benefit from additional shuffling.
Use @{tf.train.batch} or @{tf.train.shuffle_batch} if you want a
single thread producing examples to batch, or if you have a
single subgraph producing examples but you want to run it in *N* threads
(where you increase *N* until it can keep the queue full). Use
@{tf.train.batch_join} or @{tf.train.shuffle_batch_join}
if you have *N* different subgraphs producing examples to batch and you
want them run by *N* threads. Use `maybe_*` to enqueue conditionally.
* @{tf.train.batch}
* @{tf.train.maybe_batch}
* @{tf.train.batch_join}
* @{tf.train.maybe_batch_join}
* @{tf.train.shuffle_batch}
* @{tf.train.maybe_shuffle_batch}
* @{tf.train.shuffle_batch_join}
* @{tf.train.maybe_shuffle_batch_join}
|
