aboutsummaryrefslogtreecommitdiffhomepage
path: root/tensorflow/core/framework/dataset.cc
Commit message (Collapse)AuthorAge
* [tf.data] Adding support for `tf.data.AUTOTUNE` as a special value for the ↵Gravatar Jiri Simsa2018-09-17
| | | | | | | | `num_parallel_calls` argument of `tf.data.Dataset.map()`, `tf.data.Dataset.interleave()`, and `tf.contrib.data.map_and_batch()`. When `tf.data.AUTOTUNE` is specified, the level of parallelism is determined at runtime. The underlying mechanism instruments the input pipeline to build a performance model and then uses the model to find the optimal values for the parallelism knobs. PiperOrigin-RevId: 213283297
* [tf.data] Move all C++ code inside the `tensorflow::data` namespace.Gravatar Derek Murray2018-09-05
| | | | PiperOrigin-RevId: 211733735
* [tf.data] Avoiding serialization of (potentially large) tensors during ↵Gravatar Jiri Simsa2018-08-31
| | | | | | optimization. PiperOrigin-RevId: 211179990
* [tf.data] Enable optimizations for input pipelines with stateful functions.Gravatar Jiri Simsa2018-08-28
| | | | PiperOrigin-RevId: 210559796
* [tf.data] Internal refactoring of C++ classes and APIs.Gravatar Jiri Simsa2018-08-13
| | | | | | | | | | - replacing `OpKernelContext` with newly introduced `DatasetContext` in `DatasetBase` constructor to make it possible to instantiate `DatasetBase` in places where an instance of `OpKernelContext` is not available - replacing `dataset::MakeIteratorContext(OpKernelContext* ctx)` factory with `IteratorContext(OpKernelContext *ctx)` constructor. - folding `GraphDatasetBase` into `DataseBase` and removing the default implementation of `AsGraphDefInternal`, making it the responsibility of the derived class to implement it to encourage/hint developers to provide serialization logic PiperOrigin-RevId: 208560010
* [tf.data] Optimization checkpointing improvements.Gravatar Jiri Simsa2018-08-10
| | | | | | | | This CL: - changes the `OptimizeDataset` checkpointing logic to checkpoint the optimized dataset (as opposed to the original dataset + the optimizations, re-running optimization every time a checkpoint is restored) - replaces `OpKernelContext` with newly introduced `SerializationContext` in the signature of `AsGraphDefInternal` to reduce the scope of the context and also simplify the logic for overriding the `FunctionLibraryDefinition` when optimizations take place PiperOrigin-RevId: 208282562
* Making PrefetchToDevice work on XLA Compile on Demand mode. Also adds a ↵Gravatar Rohan Jain2018-08-07
| | | | | | bunch of dataset / iterator kernel registrations for XLA. PiperOrigin-RevId: 207802858
* [tf.data] Move tensorflow::dataset::MakeIteratorContext to core/frameworkGravatar Brennan Saeta2018-05-08
| | | | PiperOrigin-RevId: 195756342
* [tf.data] Move the C++ Dataset class implementations to the framework library.Gravatar Derek Murray2018-02-07
This enables the use of the `DatasetOpKernel` subclasses in custom op library code. A subsequent change will move `tf.contrib.data` kernel implementations to a custom op library. Implementation note: This change moves some classes from "tensorflow/core/graph/..." into the framework library, which does not include any code in "tensorflow/core/common_runtime/...". To break the dependency from "tensorflow/core/framework/dataset.cc" to "tensorflow/core/common_runtime/...", the `GraphDefBuilderToGraph()` method has been split out from the `GraphDefBuilder` class (where it was previously exposed as the `GraphDefBuilder::ToGraph()` utility method) and added to a new "tensorflow/core/graph/graph_def_builder_util.h" module. This method depends on ".../graph/graph_constructor.cc", which depends directly on ".../common_runtime/shape_refiner.h" and indirectly on ".../common_runtime/graph_runner.h". Since this method was used only in tests, these have been updated to point to the new utility method. PiperOrigin-RevId: 184888903