blob: 7ad4b65bd0dd995e0757278311b08de6af4d59c7 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
|
<!-- This file is machine generated: DO NOT EDIT! -->
# Data IO (Python functions)
<!-- TOC-BEGIN This section is generated by neural network: DO NOT EDIT! -->
## Contents
* [Data IO (Python Functions)](#AUTOGENERATED-data-io--python-functions-)
* [class tf.python_io.TFRecordWriter](#TFRecordWriter)
* [tf.python_io.tf_record_iterator(path)](#tf_record_iterator)
* [TFRecords Format Details](#AUTOGENERATED-tfrecords-format-details)
<!-- TOC-END This section was generated by neural network, THANKS FOR READING! -->
## Data IO (Python Functions) <div class="md-anchor" id="AUTOGENERATED-data-io--python-functions-">{#AUTOGENERATED-data-io--python-functions-}</div>
A TFRecords file represents a sequence of (binary) strings. The format is not
random access, so it is suitable for streaming large amounts of data but not
suitable if fast sharding or other non-sequential access is desired.
- - -
### class tf.python_io.TFRecordWriter <div class="md-anchor" id="TFRecordWriter">{#TFRecordWriter}</div>
A class to write records to a TFRecords file.
This class implements `__enter__` and `__exit__`, and can be used
in `with` blocks like a normal file.
- - -
#### tf.python_io.TFRecordWriter.__init__(path) {#TFRecordWriter.__init__}
Opens file `path` and creates a `TFRecordWriter` writing to it.
##### Args:
* <b>path</b>: The path to the TFRecords file.
##### Raises:
* <b>IOError</b>: If `path` cannot be opened for writing.
- - -
#### tf.python_io.TFRecordWriter.write(record) {#TFRecordWriter.write}
Write a string record to the file.
##### Args:
* <b>record</b>: str
- - -
#### tf.python_io.TFRecordWriter.close() {#TFRecordWriter.close}
Close the file.
- - -
### tf.python_io.tf_record_iterator(path) <div class="md-anchor" id="tf_record_iterator">{#tf_record_iterator}</div>
An iterator that read the records from a TFRecords file.
##### Args:
* <b>path</b>: The path to the TFRecords file.
##### Yields:
Strings.
##### Raises:
* <b>IOError</b>: If `path` cannot be opened for reading.
- - -
### TFRecords Format Details <div class="md-anchor" id="AUTOGENERATED-tfrecords-format-details">{#AUTOGENERATED-tfrecords-format-details}</div>
A TFRecords file contains a sequence of strings with CRC hashes. Each record
has the format
uint64 length
uint32 masked_crc32_of_length
byte data[length]
uint32 masked_crc32_of_data
and the records are concatenated together to produce the file. The CRC32s
are [described here](https://en.wikipedia.org/wiki/Cyclic_redundancy_check),
and the mask of a CRC is
masked_crc = ((crc >> 15) | (crc << 17)) + 0xa282ead8ul
|
