aboutsummaryrefslogtreecommitdiffhomepage
path: root/tensorflow/contrib/data
Commit message (Collapse)AuthorAge
* [tf.data] Deprecate `tf.contrib.data` and introduce `tf.data.experimental` ↵Gravatar Derek Murray2018-10-01
| | | | | | | | | | | | | | | | | | | | to replace it. This change prepares `tf.data` for TensorFlow 2.0, where `tf.contrib` will no longer exist. It retains the pre-existing endpoints in `tf.contrib.data` with deprecation warnings. Note there are some exceptions to the move: * Deprecated symbols in `tf.contrib.data` have not been moved to `tf.data.experimental`, because replacements already exist. * `tf.contrib.data.LMDBDataset` has not been moved, because we plan to move it to a SIG-maintained repository. * `tf.contrib.data.assert_element_shape()` has not yet been moved, because it depends on functionality in `tf.contrib`, and it will move in a later change. * `tf.contrib.data.AUTOTUNE` has not yet been moved, because we have not yet determined how to `tf_export()` a Python integer. * The stats-related API endpoints have not yet appeared in a released version of TensorFlow, so these are moved to `tf.data.experimental` without retaining an endpoint in `tf.contrib.data`. In addition, this change includes some build rule and ApiDef refactoring: * Some of the "//third_party/tensorflow/python:training" dependencies had to be split in order to avoid a circular dependency. * The `tf.contrib.stateless` ops now have a private core library for the generated wrappers (and accordingly are hidden in their ApiDef) so that `tf.data.experimental.sample_from_datasets()` can depend on them. PiperOrigin-RevId: 215304249
* [tf.data] Merged contrib.data's DatasetTestBase with the DatasetTestBase in ↵Gravatar Rachel Lim2018-09-28
| | | | | | core (and added that as a base class for all the contrib tests). Also changed the assertDatasetsEqual functions so they are both graph and eager compatible (took the code from CSVDatasetTest) :) PiperOrigin-RevId: 215004892
* Remove @{} api_links and ban "@{}" from python and md files.Gravatar Mark Daoust2018-09-28
| | | | PiperOrigin-RevId: 214964988
* [tf.data] Throws appropriate error while trying to checkpoint input pipeline ↵Gravatar Shivani Agrawal2018-09-28
| | | | | | with associated stats_aggregator. PiperOrigin-RevId: 214961678
* [tf.data] Move `tf.contrib.data` C++ code to a core "experimental" directory.Gravatar Derek Murray2018-09-28
| | | | | | | | NOTE: All ops and kernels previously previously defined in tensorflow/contrib/data have had their name prefixed with "Experimental" to indicate that they are not (yet) stable, and thus not subject to backwards or forwards compatibility guarantees. PiperOrigin-RevId: 214940819
* Updating the V2 variables API.Gravatar Alexandre Passos2018-09-27
| | | | PiperOrigin-RevId: 214824023
* Removing _PerDeviceGenerator and MultiDeviceIterator from contrib now that ↵Gravatar Rohan Jain2018-09-26
| | | | | | they're moved to core. I overlooked this in the CL to move to core. PiperOrigin-RevId: 214699544
* Hoisting RandomUniform out of functionsGravatar Piotr Padlewski2018-09-26
| | | | | | | This patch introduces optimization that hoists RandomUniform out of map functions. By doing it, we make function stateless, which is crucial for parallelization and vectorization. PiperOrigin-RevId: 214623178
* [tf.data] Adding serialization support for `StatsAggregatorDatasets` to make ↵Gravatar Jiri Simsa2018-09-26
| | | | | | it possible to apply static optimizations to input pipelines whose prefix contains the `set_stats_aggregator` transformation. PiperOrigin-RevId: 214619583
* Update BUILD files so that Estimator code depends on TF thru tf_no_contrib.Gravatar Michael Case2018-09-25
| | | | PiperOrigin-RevId: 214541221
* [tf.data] Adding a private method for (recursively) tracking dataset inputs.Gravatar Jiri Simsa2018-09-25
| | | | PiperOrigin-RevId: 214495925
* Automated rollback of commit 7bbd69fd16d4e97afd417786c1ee2fff27d92703Gravatar Rohan Jain2018-09-24
| | | | PiperOrigin-RevId: 214321627
* [data-stats] Exposes `StatsAggregator` and `set_stats_aggregator` in ↵Gravatar Shivani Agrawal2018-09-24
| | | | | | tf.contrib.data. PiperOrigin-RevId: 214294955
* Fix noop elimination optimization.Gravatar Piotr Padlewski2018-09-23
| | | | | | | Fix for b/116169724 Only remove noops if they refer to const nodes. PiperOrigin-RevId: 214199200
* Moving MultiDeviceIterator from contrib to core.Gravatar Rohan Jain2018-09-23
| | | | PiperOrigin-RevId: 214173896
* Fixes a bug for the case when the MultiDeviceIterator waits on backgroundGravatar Rohan Jain2018-09-21
| | | | | | thread to finish even if None is running. PiperOrigin-RevId: 214040824
* [data-stats] Collects prefetch `buffer_size` and `buffer_capacity` as ↵Gravatar Shivani Agrawal2018-09-21
| | | | | | scalar, if stats_aggregator is associated with dataset. PiperOrigin-RevId: 213989745
* [data-stats] Adds number of filtered elements as scalar summary, also adds ↵Gravatar Shivani Agrawal2018-09-20
| | | | | | number of filtered elements to monitoring counter. PiperOrigin-RevId: 213846793
* [tf.data] Use vectorization_utils::VectorizeMapDefun in MapVectorization ↵Gravatar Rachel Lim2018-09-20
| | | | | | optimization PiperOrigin-RevId: 213840320
* [tf.data] Introducing `tf.data.Dataset.window(size, shift, stride, ↵Gravatar Jiri Simsa2018-09-17
| | | | | | | | drop_remainder)`, which can be used for combining elements of input dataset into "windows". A window is itself a finite dataset and, among other things, can be used for generalized batching (see https://github.com/tensorflow/community/pull/5 for details). PiperOrigin-RevId: 213360134
* Move from deprecated self.test_session() to self.cached_session().Gravatar A. Unique TensorFlower2018-09-17
| | | | | | | | self.test_session() has been deprecated in 9962eb5e84b15e309410071b06c2ed2d6148ed44 as its name confuses readers of the test. Moving to cached_session() instead which is more explicit about: * the fact that the session may be reused. * the session is not closed even when doing a "with self.test_session()" statement. PiperOrigin-RevId: 213326581
* [tf.data] Adding support for `tf.data.AUTOTUNE` as a special value for the ↵Gravatar Jiri Simsa2018-09-17
| | | | | | | | `num_parallel_calls` argument of `tf.data.Dataset.map()`, `tf.data.Dataset.interleave()`, and `tf.contrib.data.map_and_batch()`. When `tf.data.AUTOTUNE` is specified, the level of parallelism is determined at runtime. The underlying mechanism instruments the input pipeline to build a performance model and then uses the model to find the optimal values for the parallelism knobs. PiperOrigin-RevId: 213283297
* [tf.data] Introducing an optimization that parallelizes map transformations.Gravatar Piotr Padlewski2018-09-14
| | | | | | | | Stateless MapDatasets can be paralellized by switching to ParallelMapDataset. We set `num_parallel_calls` to 2 for now, but in the future a special value will be used that result in the optimal value to be selected dynamically at runtime. This patch also exposed a memory leak which was fixed. PiperOrigin-RevId: 213015223
* [tf.data] Changes `make_batched_features_dataset` and ↵Gravatar Shivani Agrawal2018-09-13
| | | | | | `make_tf_record_dataset` default `prefetch` buffer size to auto-tune (from 1). PiperOrigin-RevId: 212900920
* Consistency in record_default shapes for tf.contrib.data.CsvDataset & ↵Gravatar Rachel Lim2018-09-13
| | | | | | | | | | | tf.decode_csv: - Modify shape assertions so that both graph and eager accept rank 0 (scalar) and rank 1 tensors as `record_defaults`, and raise an error on other shapes. - Make tests run in both graph and eager modes Fixes #22030. PiperOrigin-RevId: 212877058
* Removed `contrib.layers` dependency `bucket_by_sequence_length` tests.Gravatar A. Unique TensorFlower2018-09-13
| | | | PiperOrigin-RevId: 212829466
* Added ability to bucket without padding, as sparse tensors to ↵Gravatar A. Unique TensorFlower2018-09-12
| | | | | | `bucket_by_sequence_length`. PiperOrigin-RevId: 212684420
* [tf.data] Thread parent cancellation manager through to MapDefunOp ↵Gravatar Rachel Lim2018-09-11
| | | | | | correctly. Refactor MapDefunOp for correctness + better C++ hygiene PiperOrigin-RevId: 212589285
* [tf.data] Mechanism for collecting processing time information and modeling ↵Gravatar Jiri Simsa2018-09-11
| | | | | | performance. PiperOrigin-RevId: 212557406
* Resolving a bug where regex pattern for errors was not matching in case the ↵Gravatar A. Unique TensorFlower2018-09-10
| | | | | | error message had multiple newline characters. PiperOrigin-RevId: 212381070
* Move from deprecated self.test_session() to self.cached_session().Gravatar A. Unique TensorFlower2018-09-10
| | | | | | | | self.test_session() has been deprecated in 9962eb5e84b15e309410071b06c2ed2d6148ed44 as its name confuses readers of the test. Moving to cached_session() instead which is more explicit about: * the fact that the session may be reused. * the session is not closed even when doing a "with self.test_session()" statement. PiperOrigin-RevId: 212338134
* Move from deprecated self.test_session() to self.cached_session().Gravatar A. Unique TensorFlower2018-09-10
| | | | | | | | self.test_session() has been deprecated in 9962eb5e84b15e309410071b06c2ed2d6148ed44 as its name confuses readers of the test. Moving to cached_session() instead which is more explicit about: * the fact that the session may be reused. * the session is not closed even when doing a "with self.test_session()" statement. PiperOrigin-RevId: 212336464
* Automated rollback of commit e258e52d2c4060fc26fda43e4ce068d5ba2ab1ffGravatar Shivani Agrawal2018-09-10
| | | | PiperOrigin-RevId: 212294062
* [tf.data] Expose `tf.contrib.data.Optional` and ↵Gravatar Derek Murray2018-09-10
| | | | | | `tf.contrib.data.get_next_as_optional()`. PiperOrigin-RevId: 212263849
* [tf.data] Refactoring of optimization tests.Gravatar Jiri Simsa2018-09-08
| | | | PiperOrigin-RevId: 212119773
* Accept unknown output_shapes for MapDefun op.Gravatar Rachel Lim2018-09-07
| | | | PiperOrigin-RevId: 212045006
* Automated rollback of commit 9b15806d96cdb1ecaac1400582a01e3944b58406Gravatar Gunhan Gulsoy2018-09-07
| | | | PiperOrigin-RevId: 212008865
* [data-stats] Adds `buffer_utilization` statistics for PrefetchDataset.Gravatar Shivani Agrawal2018-09-07
| | | | | RELNOTES: n/a PiperOrigin-RevId: 211995741
* [tf.data] Adding support for `num_parallel_calls` to ↵Gravatar Jiri Simsa2018-09-06
| | | | | | | | `tf.data.Dataset.interleave`. Unlike the `tf.data.contrib.parallel_interleave` whose parallelism is tied to the `cycle_length` argument, the newly introduced `num_parallel_calls` argument of `tf.data.Dataset.interleave` is decoupled from the `cycle_length` argument and identifies the degree of parallelism to use for fetching output elements. PiperOrigin-RevId: 211886816
* [tf.data] Naming parameterized tests to facilitate invoking them ↵Gravatar Jiri Simsa2018-09-06
| | | | | | individually and using consistent style for existing test names. PiperOrigin-RevId: 211855926
* [tf.data] Move all C++ code inside the `tensorflow::data` namespace.Gravatar Derek Murray2018-09-05
| | | | PiperOrigin-RevId: 211733735
* [tf.data] Surface errors correctly in MapDefunOp by using different ↵Gravatar Rachel Lim2018-09-05
| | | | | | CancellationManagers for each run of the function. PiperOrigin-RevId: 211717580
* [tf.data] Minor fix to remove unnecessary difference between the ↵Gravatar Jiri Simsa2018-09-05
| | | | | | implementations of the batch and padded batch reducers. PiperOrigin-RevId: 211706766
* [tf.data] Add internal optimizations for executing simple functions in ↵Gravatar Derek Murray2018-09-04
| | | | | | `MapDataset`. PiperOrigin-RevId: 211520001
* [tf.data] Avoiding serialization of (potentially large) tensors during ↵Gravatar Jiri Simsa2018-08-31
| | | | | | optimization. PiperOrigin-RevId: 211179990
* [tf.data] Add special case for single dataset in ↵Gravatar Derek Murray2018-08-29
| | | | | | `tf.contrib.data.sample_from_datasets()`. PiperOrigin-RevId: 210830214
* [data-stats] Deletes deprecated `feature_stats` transformation to collect ↵Gravatar Shivani Agrawal2018-08-29
| | | | | | `features` and `feature-values` count statistics. PiperOrigin-RevId: 210828171
* [tf.data] Adds an optional label_key argument to ↵Gravatar Shivani Agrawal2018-08-29
| | | | | | `make_batch_featured_dataset()` for extracting the label from the feature dictionaries. If label_key is provided, returned dataset will be a tuple of feature dictionaries and label. PiperOrigin-RevId: 210766469
* Merge pull request #21702 from facaiy:ENH/assert_partial_shapeGravatar TensorFlower Gardener2018-08-28
|\ | | | | | | PiperOrigin-RevId: 210626817
* | [tf.data] Enable optimizations for input pipelines with stateful functions.Gravatar Jiri Simsa2018-08-28
| | | | | | | | PiperOrigin-RevId: 210559796
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-14 20:44:06 +0000