| Commit message (Collapse) | Author | Age |
|
|
|
| |
PiperOrigin-RevId: 214495925
|
|
|
|
|
|
|
|
| |
self.test_session() has been deprecated in 9962eb5e84b15e309410071b06c2ed2d6148ed44 as its name confuses readers of the test. Moving to cached_session() instead which is more explicit about:
* the fact that the session may be reused.
* the session is not closed even when doing a "with self.test_session()" statement.
PiperOrigin-RevId: 213944932
|
|
|
|
|
|
|
|
|
|
| |
- replacing `OpKernelContext` with newly introduced `DatasetContext` in `DatasetBase` constructor to make it possible to instantiate `DatasetBase` in places where an instance of `OpKernelContext` is not available
- replacing `dataset::MakeIteratorContext(OpKernelContext* ctx)` factory with `IteratorContext(OpKernelContext *ctx)` constructor.
- folding `GraphDatasetBase` into `DataseBase` and removing the default implementation of `AsGraphDefInternal`, making it the responsibility of the derived class to implement it to encourage/hint developers to provide serialization logic
PiperOrigin-RevId: 208560010
|
|
|
|
|
|
|
|
| |
This CL:
- changes the `OptimizeDataset` checkpointing logic to checkpoint the optimized dataset (as opposed to the original dataset + the optimizations, re-running optimization every time a checkpoint is restored)
- replaces `OpKernelContext` with newly introduced `SerializationContext` in the signature of `AsGraphDefInternal` to reduce the scope of the context and also simplify the logic for overriding the `FunctionLibraryDefinition` when optimizations take place
PiperOrigin-RevId: 208282562
|
|
* Add KinesisDataset support for tensorflow Dataset
This fix is an attempt to add Kinesis support
for tensorflow's Dataset. Kinesis is provided by
AWS as a managed data streaming service. It is
similiar to Apache Kafka, often used in places
where maintaining a independent Kafka cluster on AWS
is not desirable or possible.
This fix adds the Kinesis support for tensorflow Dataset.
Similiar to the Kafka integration in tensorflow,
KinesisDataset outputs tf.string for records.
Test cases have also been added, which could be invoked manually.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Expose KinesisDataset in dataset_ops.cc
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Expose KinesisDataset in python wrapper
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add test cases for KinesisDataset
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Update AWS library include files
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add Bazel BUILD files
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Rename s3_crypto to aws_crypto
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Rename with_s3_support to with_aws_support
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Selectively add kinesis to tensorflow/contrib/BUILD
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Set different partition key and pylint fix.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add missing modules in cmake's python_modules.txt
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Address review feedback
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
|