diff options
| author |  Jianwei Xie <xiejw@google.com> | 2018-03-28 14:36:18 -0700 |
|---|---|---|
| committer |  TensorFlower Gardener <gardener@tensorflow.org> | 2018-03-28 14:38:46 -0700 |
| commit | e97c9e91e016efd951dc52e82744f607d948bb2a (patch) | |
| tree | e98e3a2aaec29758533b3c331140b464ff6ce50e | |
| parent | ef6552b544b3c3bf6808be807b30dd9bd4f19669 (diff) | |
Merge changes from github.
PiperOrigin-RevId: 190835392
116 files changed, 1703 insertions, 556 deletions
diff --git a/RELEASE.md b/RELEASE.md index 6f54dee58f..c63d9f20c9 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -1,3 +1,63 @@ +# Release 1.7.0 + +## Major Features And Improvements +* Eager mode is moving out of contrib, try `tf.enable_eager_execution()`. +* Graph rewrites emulating fixed-point quantization compatible with TensorFlow Lite, supported by new `tf.contrib.quantize` package. +* Easily customize gradient computation with `tf.custom_gradient`. +* [TensorBoard Debugger Plugin](https://github.com/tensorflow/tensorboard/blob/master/tensorboard/plugins/debugger/README.md), the graphical user interface (GUI) of TensorFlow Debugger (tfdbg), is now in alpha. +* Experimental support for reading a sqlite database as a `Dataset` with new `tf.contrib.data.SqlDataset`. +* Distributed Mutex / CriticalSection added to `tf.contrib.framework.CriticalSection`. +* Better text processing with `tf.regex_replace`. +* Easy, efficient sequence input with `tf.contrib.data.bucket_by_sequence_length` + +## Bug Fixes and Other Changes +* Accelerated Linear Algebra (XLA): + * Add `MaxPoolGradGrad` support for XLA + * CSE pass from Tensorflow is now disabled in XLA. +* `tf.data`: + * `tf.data.Dataset` + * Add support for building C++ Dataset op kernels as external libraries, using the `tf.load_op_library()` mechanism. + * `Dataset.list_files()` now shuffles its output by default. + * `Dataset.shuffle(..., seed=tf.constant(0, dtype=tf.int64))` now yields the same sequence of elements as `Dataset.shuffle(..., seed=0)`. + * Add `num_parallel_reads` argument to `tf.data.TFRecordDataset`. +* `tf.contrib`: + * `tf.contrib.bayesflow.halton_sequence` now supports randomization. + * Add support for scalars in `tf.contrib.all_reduce`. + * Add `effective_sample_size` to `tf.contrib.bayesflow.mcmc_diagnostics`. + * Add `potential_scale_reduction` to `tf.contrib.bayesflow.mcmc_diagnostics`. + * Add `BatchNormalization`, `Kumaraswamy` bijectors. + * Deprecate `tf.contrib.learn`. Please check contrib/learn/README.md for instructions on how to convert existing code. + * `tf.contrib.data` + * Remove deprecated `tf.contrib.data.Dataset`, `tf.contrib.data.Iterator`, `tf.contrib.data.FixedLengthRecordDataset`, `tf.contrib.data.TextLineDataset`, and `tf.contrib.data.TFRecordDataset` classes. + * Added `bucket_by_sequence_length`, `sliding_window_batch`, and `make_batched_features_dataset` + * Remove unmaintained `tf.contrib.ndlstm`. You can find it externally at https://github.com/tmbarchive/tfndlstm. + * Moved most of `tf.contrib.bayesflow` to its own repo: `tfp` +* Other: + * tf.py_func now reports the full stack trace if an exception occurs. + * Integrate `TPUClusterResolver` with GKE's integration for Cloud TPUs. + * Add a library for statistical testing of samplers. + * Add Helpers to stream data from the GCE VM to a Cloud TPU. + * Integrate ClusterResolvers with TPUEstimator. + * Unify metropolis_hastings interface with HMC kernel. + * Move LIBXSMM convolutions to a separate --define flag so that they are disabled by default. + * Fix `MomentumOptimizer` lambda. + * Reduce `tfp.layers` boilerplate via programmable docstrings. + * Add `auc_with_confidence_intervals`, a method for computing the AUC and confidence interval with linearithmic time complexity. + * `regression_head` now accepts customized link function, to satisfy the usage that user can define their own link function if the `array_ops.identity` does not meet the requirement. + * Fix `initialized_value` and `initial_value` behaviors for `ResourceVariables` created from `VariableDef` protos. + * Add TensorSpec to represent the specification of Tensors. + * Constant folding pass is now deterministic. + * Support `float16` `dtype` in `tf.linalg.*`. + * Add `tf.estimator.export.TensorServingInputReceiver` that allows `tf.estimator.Estimator.export_savedmodel` to pass raw tensors to model functions. + +## Thanks to our Contributors + +This release contains contributions from many people at Google, as well as: + +4d55397500, Abe, Alistair Low, Andy Kernahan, Appledore, Ben, Ben Barsdell, Boris Pfahringer, Brad Wannow, Brett Koonce, Carl Thomé, cclauss, Chengzhi Chen, Chris Drake, Christopher Yeh, Clayne Robison, Codrut Grosu, Daniel Trebbien, Danny Goodman, David Goodwin, David Norman, Deron Eriksson, Donggeon Lim, Donny Viszneki, DosLin, DylanDmitri, Francisco Guerrero, Fred Reiss, gdh1995, Giuseppe, Glenn Weidner, gracehoney, Guozhong Zhuang, Haichen "Hc" Li, Harald Husum, harumitsu.nobuta, Henry Spivey, hsm207, Jekyll Song, Jerome, Jiongyan Zhang, jjsjann123, John Sungjin Park, Johnson145, JoshVarty, Julian Wolff, Jun Wang, June-One, Kamil Sindi, Kb Sriram, Kdavis-Mozilla, Kenji, lazypanda1, Liang-Chi Hsieh, Loo Rong Jie, Mahesh Bhosale, MandarJKulkarni, ManHyuk, Marcus Ong, Marshal Hayes, Martin Pool, matthieudelaro, mdfaijul, mholzel, Michael Zhou, Ming Li, Minmin Sun, Myungjoo Ham, MyungsungKwak, Naman Kamra, Peng Yu, Penghao Cen, Phil, Raghuraman-K, resec, Rohin Mohanadas, Sandeep N Gupta, Scott Tseng, seaotterman, Seo Sanghyeon, Sergei Lebedev, Ted Chang, terrytangyuan, Tim H, tkunic, Tod, vihanjain, Yan Facai (颜发才), Yin Li, Yong Tang, Yukun Chen, Yusuke Yamada + + + # Release 1.6.0 ## Breaking Changes diff --git a/configure.py b/configure.py index 22b9abedd7..0f52c0ec99 100644 --- a/configure.py +++ b/configure.py @@ -1414,7 +1414,7 @@ def main(): set_build_var(environ_cp, 'TF_NEED_S3', 'Amazon S3 File System', 'with_s3_support', True, 's3') set_build_var(environ_cp, 'TF_NEED_KAFKA', 'Apache Kafka Platform', - 'with_kafka_support', False, 'kafka') + 'with_kafka_support', True, 'kafka') set_build_var(environ_cp, 'TF_ENABLE_XLA', 'XLA JIT', 'with_xla_support', False, 'xla') set_build_var(environ_cp, 'TF_NEED_GDR', 'GDR', 'with_gdr_support', diff --git a/tensorflow/BUILD b/tensorflow/BUILD index 6ab43638ba..29a01efc84 100644 --- a/tensorflow/BUILD +++ b/tensorflow/BUILD @@ -241,6 +241,13 @@ config_setting( ) config_setting( + name = "with_kafka_support_windows_override", + define_values = {"with_kafka_support": "true"}, + values = {"cpu": "x64_windows"}, + visibility = ["//visibility:public"], +) + +config_setting( name = "with_gcp_support_android_override", define_values = {"with_gcp_support": "true"}, values = {"crosstool_top": "//external:android/crosstool"}, diff --git a/tensorflow/contrib/BUILD b/tensorflow/contrib/BUILD index bdbd738906..fb81b50fe8 100644 --- a/tensorflow/contrib/BUILD +++ b/tensorflow/contrib/BUILD @@ -51,7 +51,6 @@ py_library( "//tensorflow/contrib/image:single_image_random_dot_stereograms_py", "//tensorflow/contrib/input_pipeline:input_pipeline_py", "//tensorflow/contrib/integrate:integrate_py", - "//tensorflow/contrib/kafka", "//tensorflow/contrib/keras", "//tensorflow/contrib/kernel_methods", "//tensorflow/contrib/kfac", @@ -110,7 +109,13 @@ py_library( "//tensorflow/python:util", ] + if_mpi(["//tensorflow/contrib/mpi_collectives:mpi_collectives_py"]) + if_tensorrt([ "//tensorflow/contrib/tensorrt:init_py", - ]), + ]) + select({ + "//tensorflow:with_kafka_support_windows_override": [], + "//tensorflow:with_kafka_support": [ + "//tensorflow/contrib/kafka", + ], + "//conditions:default": [], + }), ) cc_library( @@ -120,7 +125,6 @@ cc_library( "//tensorflow/contrib/boosted_trees:boosted_trees_kernels", "//tensorflow/contrib/coder:all_kernels", "//tensorflow/contrib/data/kernels:dataset_kernels", - "//tensorflow/contrib/kafka:dataset_kernels", "//tensorflow/contrib/factorization/kernels:all_kernels", "//tensorflow/contrib/input_pipeline:input_pipeline_ops_kernels", "//tensorflow/contrib/layers:sparse_feature_cross_op_kernel", @@ -133,7 +137,13 @@ cc_library( "//tensorflow/contrib/text:all_kernels", ] + if_mpi(["//tensorflow/contrib/mpi_collectives:mpi_collectives_py"]) + if_cuda([ "//tensorflow/contrib/nccl:nccl_kernels", - ]), + ]) + select({ + "//tensorflow:with_kafka_support_windows_override": [], + "//tensorflow:with_kafka_support": [ + "//tensorflow/contrib/kafka:dataset_kernels", + ], + "//conditions:default": [], + }), ) cc_library( @@ -146,7 +156,6 @@ cc_library( "//tensorflow/contrib/factorization:all_ops", "//tensorflow/contrib/framework:all_ops", "//tensorflow/contrib/input_pipeline:input_pipeline_ops_op_lib", - "//tensorflow/contrib/kafka:dataset_ops_op_lib", "//tensorflow/contrib/layers:sparse_feature_cross_op_op_lib", "//tensorflow/contrib/nccl:nccl_ops_op_lib", "//tensorflow/contrib/nearest_neighbor:nearest_neighbor_ops_op_lib", @@ -157,7 +166,13 @@ cc_library( "//tensorflow/contrib/tensor_forest:tensor_forest_ops_op_lib", "//tensorflow/contrib/text:all_ops", "//tensorflow/contrib/tpu:all_ops", - ], + ] + select({ + "//tensorflow:with_kafka_support_windows_override": [], + "//tensorflow:with_kafka_support": [ + "//tensorflow/contrib/kafka:dataset_ops_op_lib", + ], + "//conditions:default": [], + }), ) filegroup( diff --git a/tensorflow/contrib/boosted_trees/kernels/quantile_ops.cc b/tensorflow/contrib/boosted_trees/kernels/quantile_ops.cc index 0f4c2298f5..0b28f81e7c 100644 --- a/tensorflow/contrib/boosted_trees/kernels/quantile_ops.cc +++ b/tensorflow/contrib/boosted_trees/kernels/quantile_ops.cc @@ -253,7 +253,7 @@ class CreateQuantileAccumulatorOp : public OpKernel { private: float epsilon_; int32 num_quantiles_; - // An upperbound on the number of enteries that the summaries might have + // An upper bound on the number of entries that the summaries might have // for a feature. int64 max_elements_; bool generate_quantiles_; diff --git a/tensorflow/contrib/boosted_trees/lib/utils/batch_features.cc b/tensorflow/contrib/boosted_trees/lib/utils/batch_features.cc index cf4f9a097a..35b059f349 100644 --- a/tensorflow/contrib/boosted_trees/lib/utils/batch_features.cc +++ b/tensorflow/contrib/boosted_trees/lib/utils/batch_features.cc @@ -54,7 +54,7 @@ Status BatchFeatures::Initialize( TF_CHECK_AND_RETURN_IF_ERROR( dense_float_feature.dim_size(1) == 1, errors::InvalidArgument( - "Dense float features may not be multi-valent: dim_size(1) = ", + "Dense float features may not be multivalent: dim_size(1) = ", dense_float_feature.dim_size(1))); dense_float_feature_columns_.emplace_back(dense_float_feature); } diff --git a/tensorflow/contrib/boosted_trees/lib/utils/batch_features_test.cc b/tensorflow/contrib/boosted_trees/lib/utils/batch_features_test.cc index 609519e8b1..cfe9101e74 100644 --- a/tensorflow/contrib/boosted_trees/lib/utils/batch_features_test.cc +++ b/tensorflow/contrib/boosted_trees/lib/utils/batch_features_test.cc @@ -59,7 +59,7 @@ TEST_F(BatchFeaturesTest, DenseFloatFeatures_Multivalent) { BatchFeatures batch_features(1); auto dense_vec = AsTensor<float>({3.0f, 7.0f}, {1, 2}); auto expected_error = InvalidArgument( - "Dense float features may not be multi-valent: dim_size(1) = 2"); + "Dense float features may not be multivalent: dim_size(1) = 2"); EXPECT_EQ(expected_error, batch_features.Initialize({dense_vec}, {}, {}, {}, {}, {}, {})); } diff --git a/tensorflow/contrib/boosted_trees/lib/utils/dropout_utils.cc b/tensorflow/contrib/boosted_trees/lib/utils/dropout_utils.cc index db34db998a..ce67db797d 100644 --- a/tensorflow/contrib/boosted_trees/lib/utils/dropout_utils.cc +++ b/tensorflow/contrib/boosted_trees/lib/utils/dropout_utils.cc @@ -54,7 +54,7 @@ Status DropoutUtils::DropOutTrees( if (probability_of_skipping_dropout < 0 || probability_of_skipping_dropout > 1) { return errors::InvalidArgument( - "Probability of skiping dropout must be in [0,1] range"); + "Probability of skipping dropout must be in [0,1] range"); } const auto num_trees = weights.size(); diff --git a/tensorflow/contrib/boosted_trees/lib/utils/dropout_utils.h b/tensorflow/contrib/boosted_trees/lib/utils/dropout_utils.h index 928bfbfe5c..77c16da541 100644 --- a/tensorflow/contrib/boosted_trees/lib/utils/dropout_utils.h +++ b/tensorflow/contrib/boosted_trees/lib/utils/dropout_utils.h @@ -66,7 +66,7 @@ class DropoutUtils { // Current weights and num_updates will be updated as a result of this // func std::vector<float>* current_weights, - // How many weight assignements have been done for each tree already. + // How many weight assignments have been done for each tree already. std::vector<int32>* num_updates); }; diff --git a/tensorflow/contrib/boosted_trees/lib/utils/sparse_column_iterable_test.cc b/tensorflow/contrib/boosted_trees/lib/utils/sparse_column_iterable_test.cc index 0138aae3db..cc7604745e 100644 --- a/tensorflow/contrib/boosted_trees/lib/utils/sparse_column_iterable_test.cc +++ b/tensorflow/contrib/boosted_trees/lib/utils/sparse_column_iterable_test.cc @@ -34,7 +34,7 @@ TEST_F(SparseColumnIterableTest, Empty) { } TEST_F(SparseColumnIterableTest, Iterate) { - // 8 examples having 7 sparse features with the 3rd and 7th multi-valent. + // 8 examples having 7 sparse features with the 3rd and 7th multivalent. // This can be visualized like the following: // Instance | Sparse | // 0 | x | diff --git a/tensorflow/contrib/boosted_trees/proto/tree_config.proto b/tensorflow/contrib/boosted_trees/proto/tree_config.proto index 4407c4d981..81411aa84a 100644 --- a/tensorflow/contrib/boosted_trees/proto/tree_config.proto +++ b/tensorflow/contrib/boosted_trees/proto/tree_config.proto @@ -53,7 +53,7 @@ message DenseFloatBinarySplit { // Float feature column and split threshold describing // the rule feature <= threshold. int32 feature_column = 1; - // If feature column is multivalent, this holds the index of the dimensiong + // If feature column is multivalent, this holds the index of the dimension // for the split. Defaults to 0. int32 dimension_id = 5; float threshold = 2; diff --git a/tensorflow/contrib/boosted_trees/python/kernel_tests/prediction_ops_test.py b/tensorflow/contrib/boosted_trees/python/kernel_tests/prediction_ops_test.py index c1acf35160..cf55759aaa 100644 --- a/tensorflow/contrib/boosted_trees/python/kernel_tests/prediction_ops_test.py +++ b/tensorflow/contrib/boosted_trees/python/kernel_tests/prediction_ops_test.py @@ -120,8 +120,8 @@ class PredictionOpsTest(test_util.TensorFlowTestCase): """Sets up the prediction tests. Create a batch of two examples having one dense float, two sparse float - single valued, one sparse float multidimensionl and one sparse int features. - The data looks like the following: + single valued, one sparse float multidimensional and one sparse int + features. The data looks like the following: | Instance | Dense0 | SparseF0 | SparseF1 | SparseI0 | SparseM | 0 | 7 | -3 | | 9,1 | __, 5.0 | 1 | -2 | | 4 | | 3, ___ @@ -810,7 +810,7 @@ class PredictionOpsTest(test_util.TensorFlowTestCase): # building. This tree should never be dropped. num_trees = 10 with self.test_session(): - # Empty tree ensenble. + # Empty tree ensemble. tree_ensemble_config = tree_config_pb2.DecisionTreeEnsembleConfig() # Add 10 trees with some weights. for i in range(0, num_trees): @@ -951,7 +951,7 @@ class PredictionOpsTest(test_util.TensorFlowTestCase): def testDropOutZeroProb(self): with self.test_session(): - # Empty tree ensenble. + # Empty tree ensemble. tree_ensemble_config = tree_config_pb2.DecisionTreeEnsembleConfig() # Add 1000 trees with some weights. for i in range(0, 999): @@ -994,7 +994,7 @@ class PredictionOpsTest(test_util.TensorFlowTestCase): def testAveragingAllTrees(self): with self.test_session(): - # Empty tree ensenble. + # Empty tree ensemble. tree_ensemble_config = tree_config_pb2.DecisionTreeEnsembleConfig() adjusted_tree_ensemble_config = ( tree_config_pb2.DecisionTreeEnsembleConfig()) diff --git a/tensorflow/contrib/boosted_trees/python/kernel_tests/quantile_ops_test.py b/tensorflow/contrib/boosted_trees/python/kernel_tests/quantile_ops_test.py index 81f58de28c..074623699d 100644 --- a/tensorflow/contrib/boosted_trees/python/kernel_tests/quantile_ops_test.py +++ b/tensorflow/contrib/boosted_trees/python/kernel_tests/quantile_ops_test.py @@ -482,7 +482,7 @@ class QuantilesOpTest(test_util.TensorFlowTestCase): """Sets up the quantile op tests. Create a batch of 4 examples having 2 dense and 4 sparse features. - Forth sparse feature is multivalent (3 dimensional) + Fourth sparse feature is multivalent (3 dimensional) The data looks like this | Instance | Dense 0 | Dense 1 | Sparse 0 | Sparse 1 |Sparse 2| SparseM | 0 | -0.1 | -1 | -2 | 0.1 | |_ ,1,_ diff --git a/tensorflow/contrib/boosted_trees/python/ops/quantile_ops.py b/tensorflow/contrib/boosted_trees/python/ops/quantile_ops.py index 97d57e8b23..1b184d296b 100644 --- a/tensorflow/contrib/boosted_trees/python/ops/quantile_ops.py +++ b/tensorflow/contrib/boosted_trees/python/ops/quantile_ops.py @@ -184,7 +184,7 @@ class QuantileAccumulator(saver.BaseSaverBuilder.SaveableObject): """Finalizes quantile summary stream and resets it for next iteration. Args: - stamp_token: Exepcted current token. + stamp_token: Expected current token. next_stamp_token: Next value for the token. Returns: A list of quantiles or approximate boundaries. diff --git a/tensorflow/contrib/cmake/tf_tests.cmake b/tensorflow/contrib/cmake/tf_tests.cmake index f793877c8b..92f2ab6dea 100644 --- a/tensorflow/contrib/cmake/tf_tests.cmake +++ b/tensorflow/contrib/cmake/tf_tests.cmake @@ -210,6 +210,9 @@ if (tensorflow_BUILD_PYTHON_TESTS) "${tensorflow_source_dir}/tensorflow/contrib/learn/python/learn/learn_io/graph_io_test.py" # Test is flaky on Windows GPU builds (b/38283730). "${tensorflow_source_dir}/tensorflow/contrib/factorization/python/ops/gmm_test.py" + # Disable following manual tag in BUILD. + "${tensorflow_source_dir}/tensorflow/python/keras/_impl/keras/layers/convolutional_test.py" + ) if (WIN32) set(tf_test_src_py_exclude diff --git a/tensorflow/contrib/data/python/kernel_tests/batch_dataset_op_test.py b/tensorflow/contrib/data/python/kernel_tests/batch_dataset_op_test.py index 5abb38c2d2..75482f67da 100644 --- a/tensorflow/contrib/data/python/kernel_tests/batch_dataset_op_test.py +++ b/tensorflow/contrib/data/python/kernel_tests/batch_dataset_op_test.py @@ -413,6 +413,20 @@ class BatchDatasetTest(test.TestCase): def testMapAndBatchPartialBatchDropRemainder(self): return self._testMapAndBatchPartialBatchHelper(drop_remainder=True) + def testMapAndBatchYieldsPartialBatch(self): + iterator = (dataset_ops.Dataset.range(10) + .apply(batching.map_and_batch( + lambda x: array_ops.reshape(x * x, [1]), 4)) + .make_one_shot_iterator()) + self.assertEqual([None, 1], iterator.output_shapes.as_list()) + next_element = iterator.get_next() + with self.test_session() as sess: + self.assertAllEqual([[0], [1], [4], [9]], sess.run(next_element)) + self.assertAllEqual([[16], [25], [36], [49]], sess.run(next_element)) + self.assertAllEqual([[64], [81]], sess.run(next_element)) + with self.assertRaises(errors.OutOfRangeError): + sess.run(next_element) + def testMapAndBatchSparse(self): def _sparse(i): diff --git a/tensorflow/contrib/eager/python/BUILD b/tensorflow/contrib/eager/python/BUILD index 4fba014d6f..80176397c0 100644 --- a/tensorflow/contrib/eager/python/BUILD +++ b/tensorflow/contrib/eager/python/BUILD @@ -270,7 +270,11 @@ cuda_py_test( "//tensorflow/python/eager:test", "//tensorflow/python/keras", ], - tags = ["notsan"], + tags = [ + "no_oss", # b/74395663 + "no_windows", # TODO: needs investigation on Windows + "notsan", + ], ) filegroup( diff --git a/tensorflow/contrib/eager/python/examples/spinn/spinn_test.py b/tensorflow/contrib/eager/python/examples/spinn/spinn_test.py index 9261823d77..9adf47d505 100644 --- a/tensorflow/contrib/eager/python/examples/spinn/spinn_test.py +++ b/tensorflow/contrib/eager/python/examples/spinn/spinn_test.py @@ -418,7 +418,6 @@ class SpinnTest(test_util.TensorFlowTestCase): if event.summary.value and event.summary.value[0].tag == "train/loss"] self.assertEqual(config.epochs, len(train_losses)) - self.assertLess(train_losses[-1], train_losses[0]) # 5. Verify that checkpoints exist and contains all the expected variables. self.assertTrue(glob.glob(os.path.join(config.logdir, "ckpt*"))) diff --git a/tensorflow/contrib/estimator/python/estimator/replicate_model_fn.py b/tensorflow/contrib/estimator/python/estimator/replicate_model_fn.py index e0fae2c992..fa2697800e 100644 --- a/tensorflow/contrib/estimator/python/estimator/replicate_model_fn.py +++ b/tensorflow/contrib/estimator/python/estimator/replicate_model_fn.py @@ -136,7 +136,7 @@ def replicate_model_fn(model_fn, the train_op argument of `EstimatorSpec`. loss_reduction: controls whether losses are summed or averaged. devices: Optional list of devices to replicate the model across. This - argument can be used to replice only on the subset of available GPUs. + argument can be used to replicate only on the subset of available GPUs. If `None`, then all available GPUs are going to be used for replication. If no GPUs are available, then the model is going to be placed on the CPU. diff --git a/tensorflow/contrib/factorization/kernels/clustering_ops.cc b/tensorflow/contrib/factorization/kernels/clustering_ops.cc index dd61f59585..2a6c97e8b9 100644 --- a/tensorflow/contrib/factorization/kernels/clustering_ops.cc +++ b/tensorflow/contrib/factorization/kernels/clustering_ops.cc @@ -353,7 +353,7 @@ class NearestNeighborsOp : public OpKernel { auto worker_threads = *(context->device()->tensorflow_cpu_worker_threads()); const int64 num_threads = worker_threads.num_threads; // This kernel might be configured to use fewer than the total number of - // available CPUs on the host machine. To avoid descructive interference + // available CPUs on the host machine. To avoid destructive interference // with other jobs running on the host machine, we must only use a fraction // of total available L3 cache. Unfortunately, we cannot query the host // machine to get the number of physical CPUs. So, we use a fixed per-CPU diff --git a/tensorflow/contrib/factorization/python/ops/factorization_ops.py b/tensorflow/contrib/factorization/python/ops/factorization_ops.py index 054888e734..8e0ed1d80e 100644 --- a/tensorflow/contrib/factorization/python/ops/factorization_ops.py +++ b/tensorflow/contrib/factorization/python/ops/factorization_ops.py @@ -106,7 +106,7 @@ class WALSModel(object): # the prep_gramian_op for row(column) can be run. worker_init_op = model.worker_init - # To be run once per interation sweep before the row(column) update + # To be run once per integration sweep before the row(column) update # initialize ops can be run. Note that in the distributed training # situations, this should only be run by the chief trainer. All other # trainers need to block until this is done. @@ -118,9 +118,9 @@ class WALSModel(object): init_row_update_op = model.initialize_row_update_op init_col_update_op = model.initialize_col_update_op - # Ops to upate row(column). This can either take the entire sparse tensor - # or slices of sparse tensor. For distributed trainer, each trainer - # handles just part of the matrix. + # Ops to update row(column). This can either take the entire sparse + # tensor or slices of sparse tensor. For distributed trainer, each + # trainer handles just part of the matrix. _, row_update_op, unreg_row_loss, row_reg, _ = model.update_row_factors( sp_input=matrix_slices_from_queue_for_worker_shard) row_loss = unreg_row_loss + row_reg @@ -220,7 +220,7 @@ class WALSModel(object): in the form of [[w_0, w_1, ...], [w_k, ... ], [...]], with the number of inner lists matching the number of row factor shards and the elements in each inner list are the weights for the rows of the corresponding row - factor shard. In this case, w_ij = unonbserved_weight + + factor shard. In this case, w_ij = unobserved_weight + row_weights[i] * col_weights[j]. - If this is a single non-negative real number, this value is used for all row weights and w_ij = unobserved_weight + row_weights * @@ -435,7 +435,7 @@ class WALSModel(object): gramian: Variable storing the gramian calculated from the factors. Returns: - A op that updates the gramian with the calcuated value from the factors. + A op that updates the gramian with the calculated value from the factors. """ partial_gramians = [] for f in factors: @@ -564,7 +564,7 @@ class WALSModel(object): Note that specifically this initializes the cache of the row and column weights on workers when `use_factors_weights_cache` is True. In this case, - if these weights are being calcualted and reset after the object is created, + if these weights are being calculated and reset after the object is created, it is important to ensure this ops is run afterwards so the cache reflects the correct values. """ diff --git a/tensorflow/contrib/factorization/python/ops/factorization_ops_test.py b/tensorflow/contrib/factorization/python/ops/factorization_ops_test.py index c813733915..bb5140aeb3 100644 --- a/tensorflow/contrib/factorization/python/ops/factorization_ops_test.py +++ b/tensorflow/contrib/factorization/python/ops/factorization_ops_test.py @@ -210,7 +210,7 @@ class WalsModelTest(test.TestCase): # Test row projection. # Using the specified projection weights for the 2 row feature vectors. - # This is expected to reprodue the same row factors in the model as the + # This is expected to reproduce the same row factors in the model as the # weights and feature vectors are identical to that used in model # training. projected_rows = wals_model.project_row_factors( @@ -283,8 +283,8 @@ class WalsModelTest(test.TestCase): # Test column projection. # Using the specified projection weights for the 3 column feature vectors. - # This is expected to reprodue the same column factors in the model as the - # weights and feature vectors are identical to that used in model + # This is expected to reproduce the same column factors in the model as + # the weights and feature vectors are identical to that used in model # training. projected_cols = wals_model.project_col_factors( sp_input=sp_feeder, @@ -385,7 +385,7 @@ class WalsModelTest(test.TestCase): # Test row projection. # Using the specified projection weights for the 2 row feature vectors. - # This is expected to reprodue the same row factors in the model as the + # This is expected to reproduce the same row factors in the model as the # weights and feature vectors are identical to that used in model # training. projected_rows = wals_model.project_row_factors( @@ -462,8 +462,8 @@ class WalsModelTest(test.TestCase): # Test column projection. # Using the specified projection weights for the 2 column feature vectors. - # This is expected to reprodue the same column factors in the model as the - # weights and feature vectors are identical to that used in model + # This is expected to reproduce the same column factors in the model as + # the weights and feature vectors are identical to that used in model # training. projected_cols = wals_model.project_col_factors( sp_input=sp_feeder, diff --git a/tensorflow/contrib/factorization/python/ops/gmm_ops.py b/tensorflow/contrib/factorization/python/ops/gmm_ops.py index 98d6434f47..14d4c733e3 100644 --- a/tensorflow/contrib/factorization/python/ops/gmm_ops.py +++ b/tensorflow/contrib/factorization/python/ops/gmm_ops.py @@ -280,7 +280,7 @@ class GmmAlgorithm(object): self._define_score_samples() def _define_full_covariance_probs(self, shard_id, shard): - """Defines the full covariance probabilties per example in a class. + """Defines the full covariance probabilities per example in a class. Updates a matrix with dimension num_examples X num_classes. @@ -344,7 +344,7 @@ class GmmAlgorithm(object): def _define_prior_log_prob_operation(self, shard_id): """Computes the prior probability of all samples. - Updates a vector where each item is the prior probabibility of an + Updates a vector where each item is the prior probability of an input example. Args: diff --git a/tensorflow/contrib/factorization/python/ops/gmm_test.py b/tensorflow/contrib/factorization/python/ops/gmm_test.py index 00a4734eb6..4fc9c96e9d 100644 --- a/tensorflow/contrib/factorization/python/ops/gmm_test.py +++ b/tensorflow/contrib/factorization/python/ops/gmm_test.py @@ -210,7 +210,7 @@ class GMMTestQueues(test.TestCase): return _fn # This test makes sure that there are no deadlocks when using a QueueRunner. - # Note that since cluster initialization is dependendent on inputs, if input + # Note that since cluster initialization is dependent on inputs, if input # is generated using a QueueRunner, one has to make sure that these runners # are started before the initialization. def test_queues(self): diff --git a/tensorflow/contrib/factorization/python/ops/kmeans_test.py b/tensorflow/contrib/factorization/python/ops/kmeans_test.py index 0103cc4439..88eb9cf692 100644 --- a/tensorflow/contrib/factorization/python/ops/kmeans_test.py +++ b/tensorflow/contrib/factorization/python/ops/kmeans_test.py @@ -413,7 +413,7 @@ class KMeansCosineDistanceTest(KMeansTestBase): self.assertAllClose(score, self.true_score, atol=1e-2) def test_predict_kmeans_plus_plus(self): - # Most points are concetrated near one center. KMeans++ is likely to find + # Most points are concentrated near one center. KMeans++ is likely to find # the less populated centers. points = np.array( [[2.5, 3.5], [2.5, 3.5], [-2, 3], [-2, 3], [-3, -3], [-3.1, -3.2], @@ -604,7 +604,7 @@ class KMeansTestQueues(test.TestCase): return _fn # This test makes sure that there are no deadlocks when using a QueueRunner. - # Note that since cluster initialization is dependendent on inputs, if input + # Note that since cluster initialization is dependent on inputs, if input # is generated using a QueueRunner, one has to make sure that these runners # are started before the initialization. def test_queues(self): diff --git a/tensorflow/contrib/factorization/python/ops/wals.py b/tensorflow/contrib/factorization/python/ops/wals.py index 4fe22ea26e..62db3bb4c4 100644 --- a/tensorflow/contrib/factorization/python/ops/wals.py +++ b/tensorflow/contrib/factorization/python/ops/wals.py @@ -235,7 +235,7 @@ def _wals_factorization_model_function(features, labels, mode, params): num_items: An integer, the total number of items of this axis. update_fn: A function that takes one argument (`sp_input`), and that returns a tuple of - * new_factors: A flot Tensor of the factor values after update. + * new_factors: A float Tensor of the factor values after update. * update_op: a TensorFlow op which updates the factors. * loss: A float Tensor, the unregularized loss. * reg_loss: A float Tensor, the regularization loss. diff --git a/tensorflow/contrib/learn/BUILD b/tensorflow/contrib/learn/BUILD index 9c59150580..16f80a876f 100644 --- a/tensorflow/contrib/learn/BUILD +++ b/tensorflow/contrib/learn/BUILD @@ -226,6 +226,7 @@ py_test( size = "small", srcs = ["python/learn/monitors_test.py"], srcs_version = "PY2AND3", + tags = ["no_pip_gpu"], # b/74437598 deps = [ ":learn", "//tensorflow/contrib/framework:framework_py", diff --git a/tensorflow/contrib/learn/python/learn/estimators/linear.py b/tensorflow/contrib/learn/python/learn/estimators/linear.py index 64d7ecc68e..70b70af98c 100644 --- a/tensorflow/contrib/learn/python/learn/estimators/linear.py +++ b/tensorflow/contrib/learn/python/learn/estimators/linear.py @@ -243,8 +243,8 @@ def sdca_model_fn(features, labels, mode, params): parent_scope = "linear" - with variable_scope.variable_op_scope( - features.values(), parent_scope) as scope: + with variable_scope.variable_scope( + values=features.values(), name_or_scope=parent_scope) as scope: features = features.copy() features.update(layers.transform_features(features, feature_columns)) logits, columns_to_variables, bias = ( diff --git a/tensorflow/contrib/linear_optimizer/python/sdca_estimator.py b/tensorflow/contrib/linear_optimizer/python/sdca_estimator.py index 05794a42c5..d4e54c82f9 100644 --- a/tensorflow/contrib/linear_optimizer/python/sdca_estimator.py +++ b/tensorflow/contrib/linear_optimizer/python/sdca_estimator.py @@ -140,8 +140,8 @@ def sdca_model_fn(features, labels, mode, params, config=None): parent_scope = "linear" - with variable_scope.variable_op_scope(features.values(), - parent_scope) as scope: + with variable_scope.variable_scope( + values=features.values(), name_or_scope=parent_scope) as scope: features = features.copy() features.update(layers.transform_features(features, feature_columns)) logits, columns_to_variables, bias = ( diff --git a/tensorflow/contrib/lite/README.md b/tensorflow/contrib/lite/README.md index 2680d515eb..c15ae3f233 100644 --- a/tensorflow/contrib/lite/README.md +++ b/tensorflow/contrib/lite/README.md @@ -126,6 +126,9 @@ The above pre-trained models have been trained on the ImageNet data set, which c The [TensorFlow for Poets](https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/) codelab walks through this process step-by-step. The retraining code supports retraining for both floating point and quantized inference. +# Getting started with RaspberryPi + +Using RaspberryPi can be accomplished by following the [Makefile instructions](g3doc/rpi.md). That will give a you a static library (.a) that you can build your app against. Python bindings will be coming soon as well as a demo app. ### Train a custom model A developer may choose to train a custom model using Tensorflow. TensorFlow documentation has [several tutorials](https://www.tensorflow.org/tutorials/) for building and training models. If the user has written a model using TensorFlow's Slim Framework the first step is to export this to a GraphDef file. This is necessary because Slim does not store the model structure outside the code, so to communicate with other parts of the framework it needs to be exported. Documentation for the export can be found [here](https://github.com/tensorflow/models/tree/master/research/slim#Export). The output of this step will be a .pb file for the custom model. diff --git a/tensorflow/contrib/lite/builtin_ops.h b/tensorflow/contrib/lite/builtin_ops.h index d7993e60cc..17b791e4e2 100644 --- a/tensorflow/contrib/lite/builtin_ops.h +++ b/tensorflow/contrib/lite/builtin_ops.h @@ -79,6 +79,7 @@ typedef enum { kTfLiteBuiltinBidirectionalSequenceLstm = 52, kTfLiteBuiltinCast = 53, kTfLiteBuiltinPrelu = 54, + kTfLiteBuiltinMaximum = 55, } TfLiteBuiltinOperator; #ifdef __cplusplus diff --git a/tensorflow/contrib/lite/g3doc/models.md b/tensorflow/contrib/lite/g3doc/models.md index 5b393140d6..48f43d4fc4 100644 --- a/tensorflow/contrib/lite/g3doc/models.md +++ b/tensorflow/contrib/lite/g3doc/models.md @@ -1,4 +1,4 @@ -#List of Hosted Models +# List of Hosted Models * [Inception V3 2015](https://storage.googleapis.com/download.tensorflow.org/models/tflite/inception_v3_2015_2017_11_10.zip) * [Inception V3 Slim 2016](https://storage.googleapis.com/download.tensorflow.org/models/tflite/inception_v3_slim_2016_android_2017_11_10.zip) diff --git a/tensorflow/contrib/lite/kernels/BUILD b/tensorflow/contrib/lite/kernels/BUILD index 1450c1e14b..c423c00bf5 100644 --- a/tensorflow/contrib/lite/kernels/BUILD +++ b/tensorflow/contrib/lite/kernels/BUILD @@ -156,6 +156,7 @@ cc_library( "local_response_norm.cc", "lsh_projection.cc", "lstm.cc", + "maximum.cc", "mean.cc", "mfcc.cc", "mul.cc", @@ -537,6 +538,18 @@ tf_cc_test( ) tf_cc_test( + name = "maximum_test", + size = "small", + srcs = ["maximum_test.cc"], + deps = [ + ":builtin_ops", + "//tensorflow/contrib/lite:framework", + "//tensorflow/contrib/lite/kernels:test_util", + "@com_google_googletest//:gtest", + ], +) + +tf_cc_test( name = "mean_test", size = "small", srcs = ["mean_test.cc"], diff --git a/tensorflow/contrib/lite/kernels/internal/reference/reference_ops.h b/tensorflow/contrib/lite/kernels/internal/reference/reference_ops.h index 33d60afa26..3575974ae9 100644 --- a/tensorflow/contrib/lite/kernels/internal/reference/reference_ops.h +++ b/tensorflow/contrib/lite/kernels/internal/reference/reference_ops.h @@ -404,6 +404,7 @@ inline void DepthToSpace(const T* input_data, const Dims<4>& input_dims, const int in_d = out_d + ((out_h % block_size) * block_size + out_w % block_size) * output_depth; + const int in_w = out_w / block_size; const int in_h = out_h / block_size; const int in_b = out_b; @@ -3363,6 +3364,30 @@ void TensorFlowMaximum(const T* input1_data, const Dims<4>& input1_dims, } } +template <typename T> +void TensorFlowMaximum(const T* input1_data, const Dims<4>& input1_dims, + const T* input2_data, const Dims<4>& input2_dims, + T* output_data, const Dims<4>& output_dims) { + NdArrayDesc<4> desc1; + NdArrayDesc<4> desc2; + NdArrayDescsForElementwiseBroadcast(input1_dims, input2_dims, &desc1, &desc2); + + for (int b = 0; b < ArraySize(output_dims, 3); ++b) { + for (int y = 0; y < ArraySize(output_dims, 2); ++y) { + for (int x = 0; x < ArraySize(output_dims, 1); ++x) { + for (int c = 0; c < ArraySize(output_dims, 0); ++c) { + auto out_idx = Offset(output_dims, c, x, y, b); + auto in1_idx = SubscriptToIndex(desc1, c, x, y, b); + auto in2_idx = SubscriptToIndex(desc2, c, x, y, b); + auto in1_val = input1_data[in1_idx]; + auto in2_val = input2_data[in2_idx]; + output_data[out_idx] = in1_val > in2_val ? in1_val : in2_val; + } + } + } + } +} + template <typename T1, typename T2, typename T3> void ArgMax(const T3* axis, const T1* input_data, const Dims<4>& input_dims, T2* output_data, const Dims<4>& output_dims) { diff --git a/tensorflow/contrib/lite/kernels/maximum.cc b/tensorflow/contrib/lite/kernels/maximum.cc new file mode 100644 index 0000000000..9fdf2b47ea --- /dev/null +++ b/tensorflow/contrib/lite/kernels/maximum.cc @@ -0,0 +1,106 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#include <string.h> +#include <vector> +#include "tensorflow/contrib/lite/builtin_op_data.h" +#include "tensorflow/contrib/lite/context.h" +#include "tensorflow/contrib/lite/kernels/internal/reference/reference_ops.h" +#include "tensorflow/contrib/lite/kernels/internal/tensor.h" +#include "tensorflow/contrib/lite/kernels/kernel_util.h" +#include "tensorflow/contrib/lite/kernels/op_macros.h" + +namespace tflite { +namespace ops { +namespace builtin { +namespace maximum { + +// This file has a reference implemenation of TFMaximum. +enum KernelType { + kReference, +}; + +constexpr int kInputTensor1 = 0; +constexpr int kInputTensor2 = 1; +constexpr int kOutputTensor = 0; + +struct MaximumContext { + MaximumContext(TfLiteContext* context, TfLiteNode* node) { + input1 = GetInput(context, node, kInputTensor1); + input2 = GetInput(context, node, kInputTensor2); + output = GetOutput(context, node, kOutputTensor); + } + TfLiteTensor* input1; + TfLiteTensor* input2; + TfLiteTensor* output; +}; + +TfLiteStatus Prepare(TfLiteContext* context, TfLiteNode* node) { + TF_LITE_ENSURE_EQ(context, NumInputs(node), 2); + TF_LITE_ENSURE_EQ(context, NumOutputs(node), 1); + + MaximumContext op_context(context, node); + TF_LITE_ENSURE_EQ(context, op_context.input1->type, op_context.input2->type); + TfLiteIntArray* output_dims = TfLiteIntArrayCopy(op_context.input2->dims); + op_context.output->type = op_context.input2->type; + return context->ResizeTensor(context, op_context.output, output_dims); +} + +template <KernelType kernel_type> +TfLiteStatus Eval(TfLiteContext* context, TfLiteNode* node) { + MaximumContext op_context(context, node); + +#define TF_LITE_MAXIMUM(kernel_type, data_type) \ + kernel_type::TensorFlowMaximum<data_type>( \ + GetTensorData<data_type>(op_context.input1), \ + GetTensorDims(op_context.input1), \ + GetTensorData<data_type>(op_context.input2), \ + GetTensorDims(op_context.input2), \ + GetTensorData<data_type>(op_context.output), \ + GetTensorDims(op_context.output)) + + if (kernel_type == kReference) { + switch (op_context.output->type) { + case kTfLiteFloat32: + TF_LITE_MAXIMUM(reference_ops, float); + break; + default: + context->ReportError(context, + "Type %d is currently not supported by Maximum.", + op_context.output->type); + return kTfLiteError; + } + } else { + context->ReportError(context, + "Type %d is currently not supported by Maximum.", + op_context.output->type); + return kTfLiteError; + } +#undef TF_LITE_MAXIMUM + return kTfLiteOk; +} + +} // namespace maximum + +TfLiteRegistration* Register_MAXIMUM_REF() { + static TfLiteRegistration r = {nullptr, nullptr, maximum::Prepare, + maximum::Eval<maximum::kReference>}; + return &r; +} + +TfLiteRegistration* Register_MAXIMUM() { return Register_MAXIMUM_REF(); } + +} // namespace builtin +} // namespace ops +} // namespace tflite diff --git a/tensorflow/contrib/lite/kernels/maximum_test.cc b/tensorflow/contrib/lite/kernels/maximum_test.cc new file mode 100644 index 0000000000..b3fd7d4e6f --- /dev/null +++ b/tensorflow/contrib/lite/kernels/maximum_test.cc @@ -0,0 +1,81 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#include <gtest/gtest.h> +#include "tensorflow/contrib/lite/interpreter.h" +#include "tensorflow/contrib/lite/kernels/register.h" +#include "tensorflow/contrib/lite/kernels/test_util.h" +#include "tensorflow/contrib/lite/model.h" + +namespace tflite { +namespace { + +using ::testing::ElementsAreArray; + +class MaximumOpModel : public SingleOpModel { + public: + MaximumOpModel(const TensorData& input1, const TensorData& input2, + const TensorType& output) { + input1_ = AddInput(input1); + input2_ = AddInput(input2); + output_ = AddOutput(output); + SetBuiltinOp(BuiltinOperator_MAXIMUM, BuiltinOptions_MaximumOptions, + CreateMaximumOptions(builder_).Union()); + BuildInterpreter({GetShape(input1_), GetShape(input2_)}); + } + + template <class T> + void SetInput1(std::initializer_list<T> data) { + PopulateTensor(input1_, data); + } + + template <class T> + void SetInput2(std::initializer_list<T> data) { + PopulateTensor(input2_, data); + } + + template <class T> + std::vector<T> GetOutput() { + return ExtractVector<T>(output_); + } + std::vector<int> GetOutputShape() { return GetTensorShape(output_); } + + protected: + int input1_; + int input2_; + int output_; +}; + +TEST(MaximumOpTest, FloatTest) { + std::initializer_list<float> data1 = {1.0, 0.0, -1.0, 11.0, -2.0, -1.44}; + std::initializer_list<float> data2 = {-1.0, 0.0, 1.0, 12.0, -3.0, -1.43}; + MaximumOpModel m({TensorType_FLOAT32, {3, 1, 2}}, + {TensorType_FLOAT32, {3, 1, 2}}, TensorType_FLOAT32); + m.SetInput1<float>(data1); + m.SetInput2<float>(data2); + m.Invoke(); + EXPECT_THAT(m.GetOutputShape(), ElementsAreArray({3, 1, 2})); + EXPECT_THAT( + m.GetOutput<float>(), + ElementsAreArray(ArrayFloatNear({1.0, 0.0, 1.0, 12.0, -2.0, -1.43}))); +} + +} // namespace +} // namespace tflite + +int main(int argc, char** argv) { + ::tflite::LogToStderr(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/tensorflow/contrib/lite/kernels/register.cc b/tensorflow/contrib/lite/kernels/register.cc index 62045f0a4d..0f98154b90 100644 --- a/tensorflow/contrib/lite/kernels/register.cc +++ b/tensorflow/contrib/lite/kernels/register.cc @@ -76,6 +76,7 @@ TfLiteRegistration* Register_LOG_SOFTMAX(); TfLiteRegistration* Register_CAST(); TfLiteRegistration* Register_DEQUANTIZE(); TfLiteRegistration* Register_PRELU(); +TfLiteRegistration* Register_MAXIMUM(); BuiltinOpResolver::BuiltinOpResolver() { AddBuiltin(BuiltinOperator_RELU, Register_RELU()); @@ -133,6 +134,7 @@ BuiltinOpResolver::BuiltinOpResolver() { AddBuiltin(BuiltinOperator_CAST, Register_CAST()); AddBuiltin(BuiltinOperator_DEQUANTIZE, Register_DEQUANTIZE()); AddBuiltin(BuiltinOperator_PRELU, Register_PRELU()); + AddBuiltin(BuiltinOperator_MAXIMUM, Register_MAXIMUM()); // TODO(andrewharp, ahentz): Move these somewhere more appropriate so that // custom ops aren't always included by default. diff --git a/tensorflow/contrib/lite/model.cc b/tensorflow/contrib/lite/model.cc index b7ccdf070b..791d1378f3 100644 --- a/tensorflow/contrib/lite/model.cc +++ b/tensorflow/contrib/lite/model.cc @@ -597,6 +597,9 @@ void* ParseOpData(const Operator* op, BuiltinOperator op_type, builtin_data = reinterpret_cast<void*>(params); break; } + case BuiltinOperator_MAXIMUM: { + break; + } case BuiltinOperator_DELEGATE: { // TODO(ycling): Revisit when supporting saving delegated models. error_reporter->Report("DELEGATE op shouldn't exist in model."); diff --git a/tensorflow/contrib/lite/nnapi_delegate.cc b/tensorflow/contrib/lite/nnapi_delegate.cc index e31b7c03a5..decaf9f160 100644 --- a/tensorflow/contrib/lite/nnapi_delegate.cc +++ b/tensorflow/contrib/lite/nnapi_delegate.cc @@ -350,6 +350,7 @@ void AddOpsAndParams(tflite::Interpreter* interpreter, case tflite::BuiltinOperator_DELEGATE: case tflite::BuiltinOperator_CAST: case tflite::BuiltinOperator_PRELU: + case tflite::BuiltinOperator_MAXIMUM: FATAL("Op code %d is currently not delegated to NNAPI", builtin); nn_op_type = -1; // set to invalid break; diff --git a/tensorflow/contrib/lite/python/lite.py b/tensorflow/contrib/lite/python/lite.py index 35d224924e..ed6dd036f9 100644 --- a/tensorflow/contrib/lite/python/lite.py +++ b/tensorflow/contrib/lite/python/lite.py @@ -25,9 +25,9 @@ EXPERIMENTAL: APIs here are unstable and likely to change without notice. from __future__ import absolute_import from __future__ import division from __future__ import print_function -import os -import subprocess -import tempfile +import os as _os +import subprocess as _subprocess +import tempfile as _tempfile # pylint: disable=unused-import from tensorflow.contrib.lite.python.op_hint import convert_op_hints_to_stubs @@ -74,7 +74,7 @@ else: _toco_from_proto_bin = _resource_loader.get_path_to_datafile( "../toco/python/toco_from_protos") -if _toco_from_proto_bin and not os.path.exists(_toco_from_proto_bin): +if _toco_from_proto_bin and not _os.path.exists(_toco_from_proto_bin): _toco_from_proto_bin = "toco_from_protos" @@ -102,10 +102,10 @@ def toco_convert_protos(model_flags_str, toco_flags_str, input_data_str): return _toco_python.TocoConvert( model_flags_str, toco_flags_str, input_data_str) - with tempfile.NamedTemporaryFile() as fp_toco, \ - tempfile.NamedTemporaryFile() as fp_model, \ - tempfile.NamedTemporaryFile() as fp_input, \ - tempfile.NamedTemporaryFile() as fp_output: + with _tempfile.NamedTemporaryFile() as fp_toco, \ + _tempfile.NamedTemporaryFile() as fp_model, \ + _tempfile.NamedTemporaryFile() as fp_input, \ + _tempfile.NamedTemporaryFile() as fp_output: fp_model.write(model_flags_str) fp_toco.write(toco_flags_str) fp_input.write(input_data_str) @@ -118,11 +118,11 @@ def toco_convert_protos(model_flags_str, toco_flags_str, input_data_str): fp_output.name ] cmdline = " ".join(cmd) - proc = subprocess.Popen( + proc = _subprocess.Popen( cmdline, shell=True, - stdout=subprocess.PIPE, - stderr=subprocess.STDOUT, + stdout=_subprocess.PIPE, + stderr=_subprocess.STDOUT, close_fds=True) stdout, stderr = proc.communicate() exitcode = proc.returncode diff --git a/tensorflow/contrib/lite/schema/schema.fbs b/tensorflow/contrib/lite/schema/schema.fbs index e1075971e9..7d2e00fe32 100644 --- a/tensorflow/contrib/lite/schema/schema.fbs +++ b/tensorflow/contrib/lite/schema/schema.fbs @@ -131,6 +131,7 @@ enum BuiltinOperator : byte { BIDIRECTIONAL_SEQUENCE_LSTM = 52, CAST = 53, PRELU = 54, + MAXIMUM = 55, } // Options for the builtin operators. @@ -173,6 +174,7 @@ union BuiltinOptions { LogSoftmaxOptions, CastOptions, DequantizeOptions, + MaximumOptions, } enum Padding : byte { SAME, VALID } @@ -384,6 +386,9 @@ table CastOptions { table DequantizeOptions { } +table MaximumOptions { +} + // An OperatorCode can be an enum value (BuiltinOperator) if the operator is a // builtin, or a string if the operator is custom. table OperatorCode { diff --git a/tensorflow/contrib/lite/schema/schema_generated.h b/tensorflow/contrib/lite/schema/schema_generated.h index 86daeaf5cc..66a97a1460 100755 --- a/tensorflow/contrib/lite/schema/schema_generated.h +++ b/tensorflow/contrib/lite/schema/schema_generated.h @@ -145,6 +145,9 @@ struct CastOptionsT; struct DequantizeOptions; struct DequantizeOptionsT; +struct MaximumOptions; +struct MaximumOptionsT; + struct OperatorCode; struct OperatorCodeT; @@ -255,11 +258,12 @@ enum BuiltinOperator { BuiltinOperator_BIDIRECTIONAL_SEQUENCE_LSTM = 52, BuiltinOperator_CAST = 53, BuiltinOperator_PRELU = 54, + BuiltinOperator_MAXIMUM = 55, BuiltinOperator_MIN = BuiltinOperator_ADD, - BuiltinOperator_MAX = BuiltinOperator_PRELU + BuiltinOperator_MAX = BuiltinOperator_MAXIMUM }; -inline BuiltinOperator (&EnumValuesBuiltinOperator())[53] { +inline BuiltinOperator (&EnumValuesBuiltinOperator())[54] { static BuiltinOperator values[] = { BuiltinOperator_ADD, BuiltinOperator_AVERAGE_POOL_2D, @@ -313,7 +317,8 @@ inline BuiltinOperator (&EnumValuesBuiltinOperator())[53] { BuiltinOperator_DELEGATE, BuiltinOperator_BIDIRECTIONAL_SEQUENCE_LSTM, BuiltinOperator_CAST, - BuiltinOperator_PRELU + BuiltinOperator_PRELU, + BuiltinOperator_MAXIMUM }; return values; } @@ -375,6 +380,7 @@ inline const char **EnumNamesBuiltinOperator() { "BIDIRECTIONAL_SEQUENCE_LSTM", "CAST", "PRELU", + "MAXIMUM", nullptr }; return names; @@ -425,11 +431,12 @@ enum BuiltinOptions { BuiltinOptions_LogSoftmaxOptions = 36, BuiltinOptions_CastOptions = 37, BuiltinOptions_DequantizeOptions = 38, + BuiltinOptions_MaximumOptions = 39, BuiltinOptions_MIN = BuiltinOptions_NONE, - BuiltinOptions_MAX = BuiltinOptions_DequantizeOptions + BuiltinOptions_MAX = BuiltinOptions_MaximumOptions }; -inline BuiltinOptions (&EnumValuesBuiltinOptions())[39] { +inline BuiltinOptions (&EnumValuesBuiltinOptions())[40] { static BuiltinOptions values[] = { BuiltinOptions_NONE, BuiltinOptions_Conv2DOptions, @@ -469,7 +476,8 @@ inline BuiltinOptions (&EnumValuesBuiltinOptions())[39] { BuiltinOptions_SplitOptions, BuiltinOptions_LogSoftmaxOptions, BuiltinOptions_CastOptions, - BuiltinOptions_DequantizeOptions + BuiltinOptions_DequantizeOptions, + BuiltinOptions_MaximumOptions }; return values; } @@ -515,6 +523,7 @@ inline const char **EnumNamesBuiltinOptions() { "LogSoftmaxOptions", "CastOptions", "DequantizeOptions", + "MaximumOptions", nullptr }; return names; @@ -681,6 +690,10 @@ template<> struct BuiltinOptionsTraits<DequantizeOptions> { static const BuiltinOptions enum_value = BuiltinOptions_DequantizeOptions; }; +template<> struct BuiltinOptionsTraits<MaximumOptions> { + static const BuiltinOptions enum_value = BuiltinOptions_MaximumOptions; +}; + struct BuiltinOptionsUnion { BuiltinOptions type; void *value; @@ -1016,6 +1029,14 @@ struct BuiltinOptionsUnion { return type == BuiltinOptions_DequantizeOptions ? reinterpret_cast<const DequantizeOptionsT *>(value) : nullptr; } + MaximumOptionsT *AsMaximumOptions() { + return type == BuiltinOptions_MaximumOptions ? + reinterpret_cast<MaximumOptionsT *>(value) : nullptr; + } + const MaximumOptionsT *AsMaximumOptions() const { + return type == BuiltinOptions_MaximumOptions ? + reinterpret_cast<const MaximumOptionsT *>(value) : nullptr; + } }; bool VerifyBuiltinOptions(flatbuffers::Verifier &verifier, const void *obj, BuiltinOptions type); @@ -3759,6 +3780,46 @@ inline flatbuffers::Offset<DequantizeOptions> CreateDequantizeOptions( flatbuffers::Offset<DequantizeOptions> CreateDequantizeOptions(flatbuffers::FlatBufferBuilder &_fbb, const DequantizeOptionsT *_o, const flatbuffers::rehasher_function_t *_rehasher = nullptr); +struct MaximumOptionsT : public flatbuffers::NativeTable { + typedef MaximumOptions TableType; + MaximumOptionsT() { + } +}; + +struct MaximumOptions FLATBUFFERS_FINAL_CLASS : private flatbuffers::Table { + typedef MaximumOptionsT NativeTableType; + bool Verify(flatbuffers::Verifier &verifier) const { + return VerifyTableStart(verifier) && + verifier.EndTable(); + } + MaximumOptionsT *UnPack(const flatbuffers::resolver_function_t *_resolver = nullptr) const; + void UnPackTo(MaximumOptionsT *_o, const flatbuffers::resolver_function_t *_resolver = nullptr) const; + static flatbuffers::Offset<MaximumOptions> Pack(flatbuffers::FlatBufferBuilder &_fbb, const MaximumOptionsT* _o, const flatbuffers::rehasher_function_t *_rehasher = nullptr); +}; + +struct MaximumOptionsBuilder { + flatbuffers::FlatBufferBuilder &fbb_; + flatbuffers::uoffset_t start_; + explicit MaximumOptionsBuilder(flatbuffers::FlatBufferBuilder &_fbb) + : fbb_(_fbb) { + start_ = fbb_.StartTable(); + } + MaximumOptionsBuilder &operator=(const MaximumOptionsBuilder &); + flatbuffers::Offset<MaximumOptions> Finish() { + const auto end = fbb_.EndTable(start_); + auto o = flatbuffers::Offset<MaximumOptions>(end); + return o; + } +}; + +inline flatbuffers::Offset<MaximumOptions> CreateMaximumOptions( + flatbuffers::FlatBufferBuilder &_fbb) { + MaximumOptionsBuilder builder_(_fbb); + return builder_.Finish(); +} + +flatbuffers::Offset<MaximumOptions> CreateMaximumOptions(flatbuffers::FlatBufferBuilder &_fbb, const MaximumOptionsT *_o, const flatbuffers::rehasher_function_t *_rehasher = nullptr); + struct OperatorCodeT : public flatbuffers::NativeTable { typedef OperatorCode TableType; BuiltinOperator builtin_code; @@ -3990,6 +4051,9 @@ struct Operator FLATBUFFERS_FINAL_CLASS : private flatbuffers::Table { const DequantizeOptions *builtin_options_as_DequantizeOptions() const { return builtin_options_type() == BuiltinOptions_DequantizeOptions ? static_cast<const DequantizeOptions *>(builtin_options()) : nullptr; } + const MaximumOptions *builtin_options_as_MaximumOptions() const { + return builtin_options_type() == BuiltinOptions_MaximumOptions ? static_cast<const MaximumOptions *>(builtin_options()) : nullptr; + } const flatbuffers::Vector<uint8_t> *custom_options() const { return GetPointer<const flatbuffers::Vector<uint8_t> *>(VT_CUSTOM_OPTIONS); } @@ -4168,6 +4232,10 @@ template<> inline const DequantizeOptions *Operator::builtin_options_as<Dequanti return builtin_options_as_DequantizeOptions(); } +template<> inline const MaximumOptions *Operator::builtin_options_as<MaximumOptions>() const { + return builtin_options_as_MaximumOptions(); +} + struct OperatorBuilder { flatbuffers::FlatBufferBuilder &fbb_; flatbuffers::uoffset_t start_; @@ -5696,6 +5764,29 @@ inline flatbuffers::Offset<DequantizeOptions> CreateDequantizeOptions(flatbuffer _fbb); } +inline MaximumOptionsT *MaximumOptions::UnPack(const flatbuffers::resolver_function_t *_resolver) const { + auto _o = new MaximumOptionsT(); + UnPackTo(_o, _resolver); + return _o; +} + +inline void MaximumOptions::UnPackTo(MaximumOptionsT *_o, const flatbuffers::resolver_function_t *_resolver) const { + (void)_o; + (void)_resolver; +} + +inline flatbuffers::Offset<MaximumOptions> MaximumOptions::Pack(flatbuffers::FlatBufferBuilder &_fbb, const MaximumOptionsT* _o, const flatbuffers::rehasher_function_t *_rehasher) { + return CreateMaximumOptions(_fbb, _o, _rehasher); +} + +inline flatbuffers::Offset<MaximumOptions> CreateMaximumOptions(flatbuffers::FlatBufferBuilder &_fbb, const MaximumOptionsT *_o, const flatbuffers::rehasher_function_t *_rehasher) { + (void)_rehasher; + (void)_o; + struct _VectorArgs { flatbuffers::FlatBufferBuilder *__fbb; const MaximumOptionsT* __o; const flatbuffers::rehasher_function_t *__rehasher; } _va = { &_fbb, _o, _rehasher}; (void)_va; + return tflite::CreateMaximumOptions( + _fbb); +} + inline OperatorCodeT *OperatorCode::UnPack(const flatbuffers::resolver_function_t *_resolver) const { auto _o = new OperatorCodeT(); UnPackTo(_o, _resolver); @@ -6028,6 +6119,10 @@ inline bool VerifyBuiltinOptions(flatbuffers::Verifier &verifier, const void *ob auto ptr = reinterpret_cast<const DequantizeOptions *>(obj); return verifier.VerifyTable(ptr); } + case BuiltinOptions_MaximumOptions: { + auto ptr = reinterpret_cast<const MaximumOptions *>(obj); + return verifier.VerifyTable(ptr); + } default: return false; } } @@ -6198,6 +6293,10 @@ inline void *BuiltinOptionsUnion::UnPack(const void *obj, BuiltinOptions type, c auto ptr = reinterpret_cast<const DequantizeOptions *>(obj); return ptr->UnPack(resolver); } + case BuiltinOptions_MaximumOptions: { + auto ptr = reinterpret_cast<const MaximumOptions *>(obj); + return ptr->UnPack(resolver); + } default: return nullptr; } } @@ -6356,6 +6455,10 @@ inline flatbuffers::Offset<void> BuiltinOptionsUnion::Pack(flatbuffers::FlatBuff auto ptr = reinterpret_cast<const DequantizeOptionsT *>(value); return CreateDequantizeOptions(_fbb, ptr, _rehasher).Union(); } + case BuiltinOptions_MaximumOptions: { + auto ptr = reinterpret_cast<const MaximumOptionsT *>(value); + return CreateMaximumOptions(_fbb, ptr, _rehasher).Union(); + } default: return 0; } } @@ -6514,6 +6617,10 @@ inline BuiltinOptionsUnion::BuiltinOptionsUnion(const BuiltinOptionsUnion &u) FL value = new DequantizeOptionsT(*reinterpret_cast<DequantizeOptionsT *>(u.value)); break; } + case BuiltinOptions_MaximumOptions: { + value = new MaximumOptionsT(*reinterpret_cast<MaximumOptionsT *>(u.value)); + break; + } default: break; } @@ -6711,6 +6818,11 @@ inline void BuiltinOptionsUnion::Reset() { delete ptr; break; } + case BuiltinOptions_MaximumOptions: { + auto ptr = reinterpret_cast<MaximumOptionsT *>(value); + delete ptr; + break; + } default: break; } value = nullptr; diff --git a/tensorflow/contrib/lite/testing/BUILD b/tensorflow/contrib/lite/testing/BUILD index 555ea90034..12b7b3c350 100644 --- a/tensorflow/contrib/lite/testing/BUILD +++ b/tensorflow/contrib/lite/testing/BUILD @@ -36,6 +36,7 @@ gen_zipped_test_files( "local_response_norm.zip", "log_softmax.zip", "max_pool.zip", + "maximum.zip", "mean.zip", "mul.zip", "pad.zip", diff --git a/tensorflow/contrib/lite/testing/generate_examples.py b/tensorflow/contrib/lite/testing/generate_examples.py index cb5c500136..8045052452 100644 --- a/tensorflow/contrib/lite/testing/generate_examples.py +++ b/tensorflow/contrib/lite/testing/generate_examples.py @@ -862,6 +862,41 @@ def make_log_softmax_tests(zip_path): make_zip_of_tests(zip_path, test_parameters, build_graph, build_inputs) +def make_maximum_tests(zip_path): + """Make a set of tests to do maximum.""" + + test_parameters = [{ + "input_dtype": [tf.float32], + "input_shape_1": [[3], [1, 100], [4, 2, 3], [5, 224, 224, 3]], + "input_shape_2": [[3], [1, 100], [4, 2, 3], [5, 224, 224, 3]], + }] + + def build_graph(parameters): + """Build the maximum op testing graph.""" + input_tensor_1 = tf.placeholder( + dtype=parameters["input_dtype"], + name="input_1", + shape=parameters["input_shape_1"]) + input_tensor_2 = tf.placeholder( + dtype=parameters["input_dtype"], + name="input_2", + shape=parameters["input_shape_2"]) + + out = tf.maximum(input_tensor_1, input_tensor_2) + return [input_tensor_1, input_tensor_2], [out] + + def build_inputs(parameters, sess, inputs, outputs): + values = [ + create_tensor_data(parameters["input_dtype"], + parameters["input_shape_1"]), + create_tensor_data(parameters["input_dtype"], + parameters["input_shape_2"]) + ] + return values, sess.run(outputs, feed_dict=dict(zip(inputs, values))) + + make_zip_of_tests(zip_path, test_parameters, build_graph, build_inputs) + + def make_binary_op_tests_func(binary_operator): """Return a function that does a test on a binary operator.""" return lambda zip_path: make_binary_op_tests(zip_path, binary_operator) @@ -1977,6 +2012,7 @@ def main(unused_args): "exp.zip": make_exp_tests, "log_softmax.zip": make_log_softmax_tests, "lstm.zip": make_lstm_tests, + "maximum.zip": make_maximum_tests, } out = FLAGS.zip_to_output bin_path = FLAGS.toco diff --git a/tensorflow/contrib/lite/testing/generated_examples_zip_test.cc b/tensorflow/contrib/lite/testing/generated_examples_zip_test.cc index a4a7283508..6697b86e79 100644 --- a/tensorflow/contrib/lite/testing/generated_examples_zip_test.cc +++ b/tensorflow/contrib/lite/testing/generated_examples_zip_test.cc @@ -253,6 +253,7 @@ INSTANTIATE_TESTS(l2_pool) INSTANTIATE_TESTS(l2norm) INSTANTIATE_TESTS(local_response_norm) INSTANTIATE_TESTS(log_softmax) +INSTANTIATE_TESTS(maximum) INSTANTIATE_TESTS(max_pool) INSTANTIATE_TESTS(mean) INSTANTIATE_TESTS(mul) diff --git a/tensorflow/contrib/lite/toco/tflite/operator.cc b/tensorflow/contrib/lite/toco/tflite/operator.cc index f23249cfa1..0989bfe5a3 100644 --- a/tensorflow/contrib/lite/toco/tflite/operator.cc +++ b/tensorflow/contrib/lite/toco/tflite/operator.cc @@ -863,6 +863,8 @@ std::vector<std::unique_ptr<BaseOperator>> BuildOperatorList() { ops.emplace_back(new SimpleOperator<ExpOperator>("EXP", OperatorType::kExp)); ops.emplace_back(new SimpleOperator<LogSoftmaxOperator>( "LOG_SOFTMAX", OperatorType::kLogSoftmax)); + ops.emplace_back(new SimpleOperator<TensorFlowMaximumOperator>( + "MAXIMUM", OperatorType::kTensorFlowMaximum)); return ops; } diff --git a/tensorflow/contrib/lite/toco/tflite/operator_test.cc b/tensorflow/contrib/lite/toco/tflite/operator_test.cc index 9c19f8d464..f7a213ecfc 100644 --- a/tensorflow/contrib/lite/toco/tflite/operator_test.cc +++ b/tensorflow/contrib/lite/toco/tflite/operator_test.cc @@ -109,6 +109,8 @@ TEST_F(OperatorTest, SimpleOperators) { CheckSimpleOperator<ExpOperator>("EXP", OperatorType::kExp); CheckSimpleOperator<LogSoftmaxOperator>("LOG_SOFTMAX", OperatorType::kLogSoftmax); + CheckSimpleOperator<TensorFlowMaximumOperator>( + "MAXIMUM", OperatorType::kTensorFlowMaximum); } TEST_F(OperatorTest, BuiltinAdd) { diff --git a/tensorflow/contrib/lookup/lookup_ops.py b/tensorflow/contrib/lookup/lookup_ops.py index a57a1e5421..a03e731be3 100644 --- a/tensorflow/contrib/lookup/lookup_ops.py +++ b/tensorflow/contrib/lookup/lookup_ops.py @@ -494,7 +494,7 @@ class MutableDenseHashTable(LookupInterface): value_dtype=tf.int64, default_value=-1, empty_key=0) - table.insert(keys, values) + sess.run(table.insert(keys, values)) out = table.lookup(query_keys) print(out.eval()) ``` diff --git a/tensorflow/contrib/makefile/download_dependencies.sh b/tensorflow/contrib/makefile/download_dependencies.sh index 4ae18b2cef..8b415e6527 100755 --- a/tensorflow/contrib/makefile/download_dependencies.sh +++ b/tensorflow/contrib/makefile/download_dependencies.sh @@ -34,7 +34,7 @@ PROTOBUF_URL="$(grep -o 'https://mirror.bazel.build/github.com/google/protobuf/. RE2_URL="$(grep -o 'https://mirror.bazel.build/github.com/google/re2/.*tar\.gz' "${BZL_FILE_PATH}" | head -n1)" FFT2D_URL="$(grep -o 'http.*fft\.tgz' "${BZL_FILE_PATH}" | grep -v mirror.bazel | head -n1)" ABSL_URL="$(grep -o 'https://github.com/abseil/abseil-cpp/.*tar.gz' "${BZL_FILE_PATH}" | head -n1)" -CUB_URL="$(grep -o 'https.*cub/archive.*zip' "${BZL_FILE_PATH}" | grep -v bazel-mirror | head -n1)" +CUB_URL="$(grep -o 'https.*cub/archive.*zip' "${BZL_FILE_PATH}" | grep -v mirror.bazel | head -n1)" # TODO(petewarden): Some new code in Eigen triggers a clang bug with iOS arm64, # so work around it by patching the source. diff --git a/tensorflow/contrib/makefile/tf_op_files.txt b/tensorflow/contrib/makefile/tf_op_files.txt index 5a812af4e9..7a7683c953 100644 --- a/tensorflow/contrib/makefile/tf_op_files.txt +++ b/tensorflow/contrib/makefile/tf_op_files.txt @@ -258,6 +258,7 @@ tensorflow/core/kernels/requantize.cc tensorflow/core/kernels/remote_fused_graph_execute_op.cc tensorflow/core/kernels/remote_fused_graph_execute_utils.cc tensorflow/core/kernels/batch_matmul_op_real.cc +tensorflow/core/kernels/random_op.cc tensorflow/core/ops/training_ops.cc tensorflow/core/ops/string_ops.cc tensorflow/core/ops/state_ops.cc diff --git a/tensorflow/contrib/seq2seq/kernels/beam_search_ops.cc b/tensorflow/contrib/seq2seq/kernels/beam_search_ops.cc index dfa12e873a..a9a32b7b25 100644 --- a/tensorflow/contrib/seq2seq/kernels/beam_search_ops.cc +++ b/tensorflow/contrib/seq2seq/kernels/beam_search_ops.cc @@ -74,7 +74,7 @@ class GatherTreeOp : public OpKernel { ctx, step_ids_shape.dim_size(1) == max_sequence_lengths.shape().dim_size(0), errors::InvalidArgument("batch size dimensions step_ids.shape[1] and " - "max_seqeuence_lengths.shape[0] must match. " + "max_sequence_lengths.shape[0] must match. " "but shapes are: ", step_ids_shape.DebugString(), " and ", max_sequence_lengths.shape().DebugString())); diff --git a/tensorflow/contrib/seq2seq/python/ops/attention_wrapper.py b/tensorflow/contrib/seq2seq/python/ops/attention_wrapper.py index 9ff8a343f1..be53779826 100644 --- a/tensorflow/contrib/seq2seq/python/ops/attention_wrapper.py +++ b/tensorflow/contrib/seq2seq/python/ops/attention_wrapper.py @@ -736,7 +736,7 @@ class _BaseMonotonicAttentionMechanism(_BaseAttentionMechanism): """Base attention mechanism for monotonic attention. Simply overrides the initial_alignments function to provide a dirac - distribution,which is needed in order for the monotonic attention + distribution, which is needed in order for the monotonic attention distributions to have the correct behavior. """ @@ -763,7 +763,7 @@ class _BaseMonotonicAttentionMechanism(_BaseAttentionMechanism): class BahdanauMonotonicAttention(_BaseMonotonicAttentionMechanism): """Monotonic attention mechanism with Bahadanau-style energy function. - This type of attention encorces a monotonic constraint on the attention + This type of attention enforces a monotonic constraint on the attention distributions; that is once the model attends to a given point in the memory it can't attend to any prior points at subsequence output timesteps. It achieves this by using the _monotonic_probability_fn instead of softmax to @@ -867,7 +867,7 @@ class BahdanauMonotonicAttention(_BaseMonotonicAttentionMechanism): class LuongMonotonicAttention(_BaseMonotonicAttentionMechanism): """Monotonic attention mechanism with Luong-style energy function. - This type of attention encorces a monotonic constraint on the attention + This type of attention enforces a monotonic constraint on the attention distributions; that is once the model attends to a given point in the memory it can't attend to any prior points at subsequence output timesteps. It achieves this by using the _monotonic_probability_fn instead of softmax to @@ -1133,7 +1133,7 @@ class AttentionWrapper(rnn_cell_impl.RNNCell): output_attention: Python bool. If `True` (default), the output at each time step is the attention value. This is the behavior of Luong-style attention mechanisms. If `False`, the output at each time step is - the output of `cell`. This is the beahvior of Bhadanau-style + the output of `cell`. This is the behavior of Bhadanau-style attention mechanisms. In both cases, the `attention` tensor is propagated to the next time step via the state and is used there. This flag only controls whether the attention mechanism is propagated diff --git a/tensorflow/contrib/seq2seq/python/ops/beam_search_decoder.py b/tensorflow/contrib/seq2seq/python/ops/beam_search_decoder.py index a26107b0d7..184144f64a 100644 --- a/tensorflow/contrib/seq2seq/python/ops/beam_search_decoder.py +++ b/tensorflow/contrib/seq2seq/python/ops/beam_search_decoder.py @@ -821,9 +821,9 @@ def _get_scores(log_probs, sequence_lengths, length_penalty_weight): Returns: The scores normalized by the length_penalty. """ - length_penality_ = _length_penalty( + length_penalty_ = _length_penalty( sequence_lengths=sequence_lengths, penalty_factor=length_penalty_weight) - return log_probs / length_penality_ + return log_probs / length_penalty_ def _length_penalty(sequence_lengths, penalty_factor): @@ -860,7 +860,7 @@ def _mask_probs(probs, eos_token, finished): unfinished beams remain unchanged. Args: - probs: Log probabiltiies of shape `[batch_size, beam_width, vocab_size]` + probs: Log probabilities of shape `[batch_size, beam_width, vocab_size]` eos_token: An int32 id corresponding to the EOS token to allocate probability to. finished: A boolean tensor of shape `[batch_size, beam_width]` that diff --git a/tensorflow/contrib/slim/python/slim/data/parallel_reader.py b/tensorflow/contrib/slim/python/slim/data/parallel_reader.py index b3343aef47..99ad487630 100644 --- a/tensorflow/contrib/slim/python/slim/data/parallel_reader.py +++ b/tensorflow/contrib/slim/python/slim/data/parallel_reader.py @@ -115,8 +115,8 @@ class ParallelReader(io_ops.ReaderBase): reader needs to start reading from a new file since it has finished with the previous file). - A queue runner for enqueing in the `common_queue` is automatically added to - the TF QueueRunners collection. + A queue runner for enqueuing in the `common_queue` is automatically added + to the TF QueueRunners collection. Args: queue: A Queue or a mutable string Tensor representing a handle diff --git a/tensorflow/contrib/slim/python/slim/data/prefetch_queue.py b/tensorflow/contrib/slim/python/slim/data/prefetch_queue.py index 37e9c4754c..62bd200361 100644 --- a/tensorflow/contrib/slim/python/slim/data/prefetch_queue.py +++ b/tensorflow/contrib/slim/python/slim/data/prefetch_queue.py @@ -36,9 +36,9 @@ def prefetch_queue(tensors, dynamic_pad=False, shared_name=None, name=None): - """Creates a queue to prefetech tensors from `tensors`. + """Creates a queue to prefetch tensors from `tensors`. - A queue runner for enqueing tensors into the prefetch_queue is automatically + A queue runner for enqueuing tensors into the prefetch_queue is automatically added to the TF QueueRunners collection. Example: diff --git a/tensorflow/contrib/slim/python/slim/data/tfexample_decoder.py b/tensorflow/contrib/slim/python/slim/data/tfexample_decoder.py index b3b61e1dfe..f2d31dc8db 100644 --- a/tensorflow/contrib/slim/python/slim/data/tfexample_decoder.py +++ b/tensorflow/contrib/slim/python/slim/data/tfexample_decoder.py @@ -124,7 +124,7 @@ class BoundingBox(ItemHandler): super(BoundingBox, self).__init__(self._full_keys) def tensors_to_item(self, keys_to_tensors): - """Maps the given dictionary of tensors to a contatenated list of bboxes. + """Maps the given dictionary of tensors to a concatenated list of bboxes. Args: keys_to_tensors: a mapping of TF-Example keys to parsed tensors. diff --git a/tensorflow/contrib/tensorrt/README.md b/tensorflow/contrib/tensorrt/README.md index 461e627e99..6eafc1754c 100644 --- a/tensorflow/contrib/tensorrt/README.md +++ b/tensorflow/contrib/tensorrt/README.md @@ -1,15 +1,15 @@ -Using TensorRT in TensorFlow -============================ +# Using TensorRT in TensorFlow + This module provides necessary bindings and introduces TRT_engine_op operator that wraps a subgraph in TensorRT. This is still a work in progress but should be useable with most common graphs. -Compilation ------------ +## Compilation + In order to compile the module, you need to have a local TensorRT -installation (libnvinfer.so and respective include files). During the +installation ( libnvinfer.so and respective include files ). During the configuration step, TensorRT should be enabled and installation path should be set. If installed through package managers (deb,rpm), configure script should find the necessary components from the system @@ -22,4 +22,38 @@ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/ ``` After the installation of tensorflow package, TensorRT transformation -will be available. An example use can be found in test/test_tftrt.py directory +will be available. An example use can be found in test/test_tftrt.py script + +## Installing TensorRT 3.0.4 + +In order to make use of TensorRT integration, you will need a local installation of TensorRT 3.0.4 from the [NVIDIA Developer website](https://developer.nvidia.com/tensorrt). Due to compiler compatibility, you will need to download and install the TensorRT 3.0.4 tarball for _Ubuntu 14.04_, i.e., **_TensorRT-3.0.4.Ubuntu-14.04.5.x86_64.cuda-9.0.cudnn7.0-tar.gz_**, even if you are using Ubuntu 16.04 or later. + +### Preparing TensorRT installation + +Once you have downloaded TensorRT-3.0.4.Ubuntu-14.04.5.x86_64.cuda-9.0.cudnn7.0-tar.gz, you will need to unpack it to an installation directory, which will be referred to as <install_dir>. Please replace <install_dir> with the full path of actual installation directory you choose in commands below. + +```shell +cd <install_dir> && tar -zxf /path/to/TensorRT-3.0.4.Ubuntu-14.04.5.x86_64.cuda-9.0.cudnn7.0-tar.gz +``` + +After unpacking the binaries, you have several options to use them: + +#### To run TensorFlow as a user without superuser privileges + +For a regular user without any sudo rights, you should add TensorRT to your `$LD_LIBRARY_PATH`: + + ```shell + export LD_LIBRARY_PATH=<install_dir>/TensorRT-3.0.4/lib${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} + ``` + +Then you are ready to use TensorFlow-TensorRT integration. `$LD_LIBRARY_PATH` must contain the path to TensorRT installation for TensorFlow-TensorRT integration to work. If you are using a VirtualEnv-like setup, you can add the command above to your `bin/activate` script or to your `.bashrc` script. + +#### To run TensorFlow as a superuser + + When running as a superuser, such as in a container or via sudo, the `$LD_LIBRARY_PATH` approach above may not work. The following is preferred when the user has superuser privileges: + + ```shell + echo "<install_dir>/TensorRT-3.0.4/lib" | sudo tee /etc/ld.so.conf.d/tensorrt304.conf && sudo ldconfig + ``` + + Please ensure that any existing deb package installation of TensorRT is removed before following these instructions to avoid package conflicts.
\ No newline at end of file diff --git a/tensorflow/contrib/tensorrt/convert/convert_graph.cc b/tensorflow/contrib/tensorrt/convert/convert_graph.cc index eea8c8efa2..ff8cc6374d 100644 --- a/tensorflow/contrib/tensorrt/convert/convert_graph.cc +++ b/tensorflow/contrib/tensorrt/convert/convert_graph.cc @@ -49,12 +49,13 @@ namespace tensorrt { namespace convert { namespace { -bool IsTensorRTCandidate(const tensorflow::NodeDef& node_def) { +bool IsTensorRTCandidate(const tensorflow::Node* node) { // LINT.IfChange // TODO(jie): Segmentation shouldn't associated with op name. // Split it into a registration for each kernel. static const std::set<string> candidate_ops = { "Identity", + "Snapshot", "Const", "Conv2D", "MaxPool", @@ -74,7 +75,7 @@ bool IsTensorRTCandidate(const tensorflow::NodeDef& node_def) { // TODO(ben,jie): ... }; // LINT.ThenChange(//tensorflow/contrib/tensorrt/convert/convert_nodes.h) - return candidate_ops.count(node_def.op()); + return candidate_ops.count(node->type_string()); } void GetSubGraphIncomingEdges(const tensorflow::Graph& graph, @@ -84,10 +85,10 @@ void GetSubGraphIncomingEdges(const tensorflow::Graph& graph, const tensorflow::Node* node = graph.FindNodeId(node_id); for (const tensorflow::Edge* edge : node->in_edges()) { if (!subgraph_node_ids.count(edge->src()->id()) && - !edge->src()->IsSource()) { + !edge->src()->IsSource() && !edge->IsControlEdge()) { incoming_edges->insert(edge); } else { - VLOG(2) << edge->src()->name() << " N, "; + VLOG(2) << node->name() << " -> " << edge->src()->name() << " N, "; } } } @@ -100,11 +101,11 @@ void GetSubGraphOutgoingEdges(const tensorflow::Graph& graph, const tensorflow::Node* node = graph.FindNodeId(node_id); for (const tensorflow::Edge* edge : node->out_edges()) { if (!subgraph_node_ids.count(edge->dst()->id()) && - !edge->dst()->IsSink()) { - VLOG(2) << edge->dst()->name() << " Y, "; + !edge->dst()->IsSink() && !edge->IsControlEdge()) { + VLOG(2) << node->name() << " -> " << edge->dst()->name() << " Y, "; outgoing_edges->insert(edge); } else { - VLOG(2) << edge->dst()->name() << " N, "; + VLOG(2) << node->name() << " -> " << edge->dst()->name() << " N, "; } } } @@ -409,8 +410,9 @@ tensorflow::Status ConvertGraphDefToTensorRT( tensorflow::Status status = ConvertSubGraphToTensorRT(&p); if (status != tensorflow::Status::OK()) { LOG(WARNING) << "subgraph conversion error for subgraph_index:" << count - << " due to: \n" - << status.ToString() << " SKIPPING......"; + << " due to: \"" << status.ToString() + << "\" SKIPPING......( " << subgraph_node_names.size() + << " nodes)"; } count++; } diff --git a/tensorflow/contrib/tensorrt/convert/convert_nodes.cc b/tensorflow/contrib/tensorrt/convert/convert_nodes.cc index 92a692baa7..370911e4d9 100644 --- a/tensorflow/contrib/tensorrt/convert/convert_nodes.cc +++ b/tensorflow/contrib/tensorrt/convert/convert_nodes.cc @@ -53,8 +53,8 @@ limitations under the License. namespace tensorflow { namespace tensorrt { namespace convert { +using ::tensorflow::strings::StrAppend; using ::tensorflow::strings::StrCat; - namespace { inline tensorflow::Status ConvertDType(tensorflow::DataType tf_dtype, @@ -430,9 +430,8 @@ class Converter { tensorflow::tensorrt::TRTWeightStore* weight_store_; bool fp16_; void register_op_converters(); - std::vector<TRT_TensorOrWeights> get_inputs( - const tensorflow::NodeDef& node_def) { - std::vector<TRT_TensorOrWeights> inputs; + tensorflow::Status get_inputs(const tensorflow::NodeDef& node_def, + std::vector<TRT_TensorOrWeights>* inputs) { for (auto const& input_name : node_def.input()) { /************************************************************************* * TODO(jie) handle case 1) here @@ -453,13 +452,17 @@ class Converter { VLOG(2) << "retrieve input: " << name; if (trt_tensors_.count(name)) { - inputs.push_back(trt_tensors_.at(name)); + inputs->push_back(trt_tensors_.at(name)); } else { - LOG(FATAL) << "input: " << name << " not available for node at, " - << node_def.name(); + string str("Node "); + StrAppend(&str, node_def.name(), " should have an input named '", name, + "' but it is not available"); + LOG(WARNING) << "input: " << name << " not available for node at " + << node_def.name(); + return tensorflow::errors::InvalidArgument(str); } } - return inputs; + return tensorflow::Status::OK(); } public: @@ -483,7 +486,8 @@ class Converter { } tensorflow::Status convert_node(const tensorflow::NodeDef& node_def) { - std::vector<TRT_TensorOrWeights> inputs = this->get_inputs(node_def); + std::vector<TRT_TensorOrWeights> inputs; + TF_RETURN_IF_ERROR(this->get_inputs(node_def, &inputs)); string op = node_def.op(); if (!op_registry_.count(op)) { return tensorflow::errors::Unimplemented( @@ -548,6 +552,19 @@ class Converter { } }; +TRT_ShapedWeights ConvertFP32ToFP16(Converter& ctx, + const TRT_ShapedWeights& weights_src) { + auto dtype_new = tensorflow::DataType::DT_HALF; + TRT_ShapedWeights weights = + ctx.get_temp_weights(dtype_new, weights_src.shape_); + const float* src = static_cast<const float*>(weights_src.GetValues()); + Eigen::half* dst = const_cast<Eigen::half*>( + static_cast<Eigen::half const*>(weights.GetValues())); + for (int64_t i = 0; i < weights_src.count(); i++) { + dst[i] = Eigen::half_impl::float_to_half_rtne(src[i]); + } + return weights; +} // **************************************************************************** // Constant folding functions // TODO(jie): once optimizer kicks in, we should have done constant folding @@ -875,7 +892,7 @@ tensorflow::Status BinaryTensorOpWeight( // Check type consistency nvinfer1::DataType ttype; - TF_CHECK_OK(ConvertDType(weights.type_, &ttype)); + TF_RETURN_IF_ERROR(ConvertDType(weights.type_, &ttype)); // Check scale mode auto dims_w = weights.shape_; @@ -957,6 +974,10 @@ tensorflow::Status BinaryTensorOpWeight( } } + if (ctx.isFP16()) { + weights = ConvertFP32ToFP16(ctx, weights); + } + // prepare weights TRT_ShapedWeights shift_weights(weights.type_); TRT_ShapedWeights scale_weights(weights.type_); @@ -998,9 +1019,7 @@ enum class ConvolutionType { DEFAULT, DEPTHWISE_CONV }; tensorflow::Status ConvertConv2DHelper( Converter& ctx, const tensorflow::NodeDef& node_def, const std::vector<TRT_TensorOrWeights>& inputs, - std::vector<TRT_TensorOrWeights>* outputs, - int group // group ==0 specifies depthwise conv -) { + std::vector<TRT_TensorOrWeights>* outputs, int group) { const nvinfer1::ITensor* tensor = inputs.at(0).tensor(); TFAttrs attrs(node_def); @@ -1025,6 +1044,10 @@ tensorflow::Status ConvertConv2DHelper( VLOG(2) << "groups count: " << num_groups; TRT_ShapedWeights weights_rsck = inputs.at(1).weights(); + if (ctx.isFP16()) { + weights_rsck = ConvertFP32ToFP16(ctx, inputs.at(1).weights()); + } + TRT_ShapedWeights weights = ctx.get_temp_weights_like(weights_rsck); ReorderRSCKToKCRS(weights_rsck, &weights, num_groups); TRT_ShapedWeights biases(weights.type_); @@ -1134,9 +1157,9 @@ tensorflow::Status BinaryTensorOpTensor( CHECK_EQ_TYPE(tensor_r->getType(), dtype); auto op_pair = ops.find(node_def.op()); if (op_pair == ops.end()) - return tensorflow::errors::Unimplemented("binary op: " + node_def.op() + - " not supported at: " + - node_def.name()); + return tensorflow::errors::Unimplemented( + "binary op: " + node_def.op() + + " not supported at: " + node_def.name()); nvinfer1::IElementWiseLayer* layer = ctx.network()->addElementWise( *const_cast<nvinfer1::ITensor*>(tensor_l), @@ -1295,8 +1318,11 @@ tensorflow::Status ConvertScale(Converter& ctx, // Implement tensor binaryOp weight [channel wise] for now; const nvinfer1::ITensor* tensor = inputs.at(0).tensor(); - // TODO(jie): handle NHWC/NCHW transpose; TRT_ShapedWeights weights = inputs.at(1).weights(); + if (ctx.isFP16()) { + weights = ConvertFP32ToFP16(ctx, inputs.at(1).weights()); + } + TRT_ShapedWeights empty_weights(weights.type_); TFAttrs attrs(node_def); @@ -1376,8 +1402,11 @@ tensorflow::Status ConvertConst(Converter& ctx, scalar_shape.d[0] = weights_tensor.float_val_size(); scalar_shape.type[0] = nvinfer1::DimensionType::kSPATIAL; } else { - LOG(FATAL) << "Broadcast on weights only supports kCHANNEL and" - << " kUNIFORM, at: " << node_def.name(); + LOG(WARNING) << "Broadcast on weights only supports kCHANNEL and" + << " kUNIFORM, at: " << node_def.name(); + string err_str("Broadcast method is not supported for '"); + StrAppend(&err_str, node_def.name(), "' of type ", node_def.op()); + return tensorflow::errors::InvalidArgument(err_str); } } } else { @@ -1391,33 +1420,16 @@ tensorflow::Status ConvertConst(Converter& ctx, scalar_shape.type[i] = nvinfer1::DimensionType::kSPATIAL; } } - if (ctx.isFP16()) { - auto dtype_new = tensorflow::DataType::DT_HALF; - size_t len_data = tensorflow::DataTypeSize(dtype_new); - for (int i = 0; i < scalar_shape.nbDims; i++) - len_data *= scalar_shape.d[i]; - ctx.weight_store()->store_.push_back(std::vector<uint8_t>(len_data)); - void* dst = static_cast<void*>(&(ctx.weight_store()->store_.back()[0])); - tensorflow::Tensor temp_tensor(tensorflow::DT_HALF, tensor.shape()); - auto half_tensor = temp_tensor.flat<Eigen::half>(); - Eigen::DefaultDevice defd; - half_tensor.device(defd) = - tensor.flat<float>().template cast<Eigen::half>(); - memcpy(dst, half_tensor.data(), len_data); // store into weight store - weights = TRT_ShapedWeights(dtype_new, dst, scalar_shape); - } else { - size_t len_data = tensorflow::DataTypeSize(dtype); - for (int i = 0; i < scalar_shape.nbDims; i++) - len_data *= scalar_shape.d[i]; - ctx.weight_store()->store_.push_back(std::vector<uint8_t>(len_data)); - void* dst = static_cast<void*>(&(ctx.weight_store()->store_.back()[0])); - std::vector<float> tensor_data( - weights_tensor.float_val().begin(), - weights_tensor.float_val() - .end()); // make a local copy first to flatten - memcpy(dst, tensor_data.data(), len_data); // store into weight store - weights = TRT_ShapedWeights(dtype, dst, scalar_shape); - } + size_t len_data = tensorflow::DataTypeSize(dtype); + for (int i = 0; i < scalar_shape.nbDims; i++) len_data *= scalar_shape.d[i]; + ctx.weight_store()->store_.push_back(std::vector<uint8_t>(len_data)); + void* dst = static_cast<void*>(&(ctx.weight_store()->store_.back()[0])); + std::vector<float> tensor_data( + weights_tensor.float_val().begin(), + weights_tensor.float_val() + .end()); // make a local copy first to flatten + memcpy(dst, tensor_data.data(), len_data); // store into weight store + weights = TRT_ShapedWeights(dtype, dst, scalar_shape); } else if (!weights_tensor.int_val().empty()) { VLOG(2) << "int!!!" << node_def.name(); nvinfer1::Dims scalar_shape; @@ -1432,8 +1444,11 @@ tensorflow::Status ConvertConst(Converter& ctx, scalar_shape.d[0] = weights_tensor.int_val_size(); scalar_shape.type[0] = nvinfer1::DimensionType::kSPATIAL; } else { - LOG(FATAL) << "Broadcast on weights only supports kCHANNEL and" - << " kUNIFORM, at: " << node_def.name(); + LOG(WARNING) << "Broadcast on weights only supports kCHANNEL and" + << " kUNIFORM, at: " << node_def.name(); + string err_str("Broadcast method is not supported for '"); + StrAppend(&err_str, node_def.name(), "' of type ", node_def.op()); + return tensorflow::errors::InvalidArgument(err_str); } } } else { @@ -1447,62 +1462,23 @@ tensorflow::Status ConvertConst(Converter& ctx, scalar_shape.type[i] = nvinfer1::DimensionType::kSPATIAL; } } - if (ctx.isFP16()) { - auto dtype_new = tensorflow::DataType::DT_HALF; - size_t len_data = tensorflow::DataTypeSize(dtype_new); - for (int i = 0; i < scalar_shape.nbDims; i++) - len_data *= scalar_shape.d[i]; - ctx.weight_store()->store_.push_back(std::vector<uint8_t>(len_data)); - void* dst = static_cast<void*>(&(ctx.weight_store()->store_.back()[0])); - tensorflow::Tensor temp_tensor(tensorflow::DT_HALF, tensor.shape()); - TTypes<Eigen::half>::Flat half_tensor = temp_tensor.flat<Eigen::half>(); - Eigen::DefaultDevice defd; - switch (dtype) { - case (tensorflow::DT_INT32): { - half_tensor.device(defd) = - tensor.flat<int32>().template cast<Eigen::half>(); - break; - } - case (tensorflow::DT_INT16): { - half_tensor.device(defd) = - tensor.flat<int16>().template cast<Eigen::half>(); - break; - } - case (tensorflow::DT_INT8): { - half_tensor.device(defd) = - tensor.flat<int8>().template cast<Eigen::half>(); - break; - } - case (tensorflow::DT_UINT8): { - half_tensor.device(defd) = - tensor.flat<uint8>().template cast<Eigen::half>(); - break; - } - default: - return tensorflow::errors::InvalidArgument( - "Datatype " + tensorflow::DataTypeString(dtype) + - " for FP16 conversion"); - break; - }; - memcpy(dst, half_tensor.data(), len_data); // store into weight store - weights = TRT_ShapedWeights(dtype_new, dst, scalar_shape); - } else { - size_t len_data = tensorflow::DataTypeSize(dtype); - for (int i = 0; i < scalar_shape.nbDims; i++) - len_data *= scalar_shape.d[i]; - size_t len_tensor = weights_tensor.int_val_size() * sizeof(int32); - len_data = std::max(len_data, len_tensor); - ctx.weight_store()->store_.push_back(std::vector<uint8_t>(len_data)); - void* dst = static_cast<void*>(&(ctx.weight_store()->store_.back()[0])); - std::vector<int32> tensor_data( - weights_tensor.int_val().begin(), - weights_tensor.int_val() - .end()); // make a local copy first to flatten - // doesn't have to be contiguous - memcpy(dst, tensor_data.data(), len_tensor); // store into weight store - weights = TRT_ShapedWeights(dtype, dst, scalar_shape); - } + // we should not have converted //if (ctx.isFP16()) { + size_t len_data = tensorflow::DataTypeSize(dtype); + for (int i = 0; i < scalar_shape.nbDims; i++) len_data *= scalar_shape.d[i]; + size_t len_tensor = weights_tensor.int_val_size() * sizeof(int32); + len_data = std::max(len_data, len_tensor); + ctx.weight_store()->store_.push_back(std::vector<uint8_t>(len_data)); + void* dst = static_cast<void*>(&(ctx.weight_store()->store_.back()[0])); + std::vector<int32> tensor_data( + weights_tensor.int_val().begin(), + weights_tensor.int_val().end()); // make a local copy first to flatten + // doesn't have to be contigous + memcpy(dst, tensor_data.data(), len_tensor); // store into weight store + weights = TRT_ShapedWeights(dtype, dst, scalar_shape); } else if (!weights_tensor.tensor_content().empty()) { + // obsolete method. + // After optimization path, we do not see weights in this format. + // fp16 conversion technically should be needed here. VLOG(2) << "TENSOR!!!" << node_def.name(); const auto& content = weights_tensor.tensor_content(); @@ -1784,8 +1760,6 @@ tensorflow::Status ConvertConcat(Converter& ctx, TRT_ShapedWeights axis = inputs.at(input_size).weights(); TFAttrs attrs(node_def); - // auto attr_size = attrs.at("N")->i(); - // auto data_type = attrs.get<nvinfer1::DataType>("T"); auto index_type = attrs.get<tensorflow::DataType>("Tidx"); // TODO(jie): handle data type @@ -1875,71 +1849,103 @@ tensorflow::Status ConvertFusedBatchNorm( "only is_training=false is supported, at " + node_def.name()); } nvinfer1::ITensor const* tensor = inputs.at(0).tensor(); - TRT_ShapedWeights scale_weights = inputs.at(1).weights(); - TRT_ShapedWeights offset_weights = inputs.at(2).weights(); - TRT_ShapedWeights mean_weights = inputs.at(3).weights(); - TRT_ShapedWeights variance_weights = inputs.at(4).weights(); - TRT_ShapedWeights dummy_power_weights(scale_weights.type_); - TRT_ShapedWeights combined_scale_weights = - ctx.get_temp_weights_like(scale_weights); - TRT_ShapedWeights combined_offset_weights = - ctx.get_temp_weights_like(offset_weights); - size_t nweight = scale_weights.count(); - if ((scale_weights.type_ == offset_weights.type_) && - (mean_weights.type_ == variance_weights.type_) && - (scale_weights.type_ == variance_weights.type_)) { - if ((scale_weights.type_ != tensorflow::DataType::DT_FLOAT) && - (scale_weights.type_ != tensorflow::DataType::DT_HALF)) { + + // Check parameter types + auto parameter_type = inputs.at(1).weights().type_; + if ((parameter_type != tensorflow::DataType::DT_FLOAT) && + (parameter_type != tensorflow::DataType::DT_HALF)) { + return tensorflow::errors::Unimplemented( + "only float32 or float16 weight data type is supported, for node " + + node_def.name() + " got " + tensorflow::DataTypeString(parameter_type)); + } + for (int i = 1; i < 5; i++) { + if (inputs.at(i).weights().type_ != parameter_type) { return tensorflow::errors::Unimplemented( - "only float32 or float16 weight data type is supported, for node " + - node_def.name() + " got " + - tensorflow::DataTypeString(scale_weights.type_)); + "Inconsistent parameter type for batchnormis not supported, at: " + + node_def.name()); } - if (scale_weights.type_ == tensorflow::DT_FLOAT) { - for (size_t i = 0; i < nweight; ++i) { - float scale = (static_cast<float const*>(scale_weights.GetValues()))[i]; - float offset = - (static_cast<float const*>(offset_weights.GetValues()))[i]; - float mean = (static_cast<float const*>(mean_weights.GetValues()))[i]; - float variance = - (static_cast<float const*>(variance_weights.GetValues()))[i]; - float& combined_scale_ref = const_cast<float*>( - static_cast<float const*>(combined_scale_weights.GetValues()))[i]; - float& combined_offset_ref = const_cast<float*>( - static_cast<float const*>(combined_offset_weights.GetValues()))[i]; - combined_scale_ref = scale / sqrtf(variance + epsilon); - combined_offset_ref = offset - mean * combined_scale_ref; - } - } else { - const Eigen::half* scale_vals = - (static_cast<Eigen::half const*>(scale_weights.GetValues())); - const Eigen::half* off_vals = - (static_cast<Eigen::half const*>(offset_weights.GetValues())); - const Eigen::half* mean_vals = - (static_cast<Eigen::half const*>(mean_weights.GetValues())); - const Eigen::half* variance_vals = - (static_cast<Eigen::half const*>(variance_weights.GetValues())); - Eigen::half* comb_scale_vals = const_cast<Eigen::half*>( - static_cast<Eigen::half const*>(combined_scale_weights.GetValues())); - Eigen::half* comb_off_vals = const_cast<Eigen::half*>( - static_cast<Eigen::half const*>(combined_offset_weights.GetValues())); - for (size_t i = 0; i < nweight; ++i) { - float scale(scale_vals[i]); - float offset(off_vals[i]); - float mean(mean_vals[i]); - float variance(variance_vals[i]); - float combined_scale_ref = scale / sqrtf(variance + epsilon); - comb_scale_vals[i] = Eigen::half(combined_scale_ref); - float combined_offset_ref = offset - mean * combined_scale_ref; - comb_off_vals[i] = Eigen::half(combined_offset_ref); + } + + TRT_ShapedWeights dummy_power_weights(parameter_type); + size_t nweight = 0; + for (int i = 1; i < 5; i++) { + nweight = std::max(nweight, (size_t)inputs.at(i).weights().count()); + } + TRT_ShapedWeights* ptr_shape_weights = nullptr; + for (int i = 1; i < 5; i++) { + if (inputs.at(i).weights().count() == nweight) { + ptr_shape_weights = + const_cast<TRT_ShapedWeights*>(&(inputs.at(i).weights())); + } else if (inputs.at(i).weights().count() != 1) { + return tensorflow::errors::InvalidArgument( + "Inconsistent batchnorm parameter count, at: " + node_def.name()); + } + } + // We could technically have two weights with different shape. + // that requires two addScale op, arguably less performant + TRT_ShapedWeights combined_scale_weights = + ctx.get_temp_weights_like(*ptr_shape_weights); + TRT_ShapedWeights combined_offset_weights = + ctx.get_temp_weights_like(*ptr_shape_weights); + + const Eigen::half* cast_vals_array[4]; + const float* vals_array[4]; + for (int j = 0; j < 4; j++) { + cast_vals_array[j] = + static_cast<Eigen::half const*>(inputs.at(j + 1).weights().GetValues()); + vals_array[j] = + static_cast<float const*>(inputs.at(j + 1).weights().GetValues()); + } + Eigen::half* cast_combined_scale_vals = const_cast<Eigen::half*>( + static_cast<Eigen::half const*>(combined_scale_weights.GetValues())); + Eigen::half* cast_combined_offset_vals = const_cast<Eigen::half*>( + static_cast<Eigen::half const*>(combined_offset_weights.GetValues())); + float* combined_scale_vals = const_cast<float*>( + static_cast<float const*>(combined_scale_weights.GetValues())); + float* combined_offset_vals = const_cast<float*>( + static_cast<float const*>(combined_offset_weights.GetValues())); + + for (size_t i = 0; i < nweight; ++i) { + float batchnorm_data[4]; + for (int j = 0; j < 4; j++) { + if (inputs.at(j + 1).weights().count() != 1) { + if (parameter_type == tensorflow::DT_FLOAT) { + batchnorm_data[j] = vals_array[j][i]; + } else if (parameter_type == tensorflow::DT_HALF) { + batchnorm_data[j] = + Eigen::half_impl::half_to_float(cast_vals_array[j][i]); + } + } else { + if (parameter_type == tensorflow::DT_FLOAT) { + batchnorm_data[j] = vals_array[j][0]; + } else if (parameter_type == tensorflow::DT_HALF) { + batchnorm_data[j] = + Eigen::half_impl::half_to_float(cast_vals_array[j][0]); + } } } + float scale = batchnorm_data[0]; + float offset = batchnorm_data[1]; + float mean = batchnorm_data[2]; + float variance = batchnorm_data[3]; + float combined_scale_val = scale / sqrtf(variance + epsilon); + float combined_offset_val = offset - mean * combined_scale_val; + if (parameter_type == tensorflow::DT_FLOAT) { + combined_scale_vals[i] = combined_scale_val; + combined_offset_vals[i] = combined_offset_val; + } else if (parameter_type == tensorflow::DT_HALF) { + cast_combined_scale_vals[i] = Eigen::half(combined_scale_val); + cast_combined_offset_vals[i] = Eigen::half(combined_offset_val); + } } - nvinfer1::IScaleLayer* layer = ctx.network()->addScale( - *const_cast<nvinfer1::ITensor*>(tensor), nvinfer1::ScaleMode::kCHANNEL, - combined_offset_weights.GetWeightsForTRT(), - combined_scale_weights.GetWeightsForTRT(), - dummy_power_weights.GetWeightsForTRT()); + + nvinfer1::ScaleMode mode = nweight == 1 ? nvinfer1::ScaleMode::kUNIFORM + : nvinfer1::ScaleMode::kCHANNEL; + nvinfer1::IScaleLayer* layer = + ctx.network()->addScale(*const_cast<nvinfer1::ITensor*>(tensor), mode, + combined_offset_weights.GetWeightsForTRT(), + combined_scale_weights.GetWeightsForTRT(), + dummy_power_weights.GetWeightsForTRT()); nvinfer1::ITensor* output_tensor = layer->getOutput(0); outputs->push_back(TRT_TensorOrWeights(output_tensor)); return tensorflow::Status::OK(); @@ -2050,6 +2056,7 @@ void Converter::register_op_converters() { op_registry_["Const"] = ConvertConst; // TODO(ben,jie): this is a temp hack. op_registry_["Identity"] = ConvertIdentity; // Identity should be removed + op_registry_["Snapshot"] = ConvertIdentity; // Snapshot should be removed // resnet_50_v1 slim implementation op_registry_["Add"] = ConvertBinary; @@ -2143,8 +2150,11 @@ tensorflow::Status ConvertCalibrationNodeToEngineNode( calib_res->thr_->join(); delete calib_res->thr_; if (!calib_res->engine_) { - LOG(FATAL) << "Calibration failed!, engine is nullptr. Did you run " + LOG(ERROR) << "Calibration failed!, engine does not exist. Did you run " "calibration graph?"; + return tensorflow::errors::FailedPrecondition( + "Calibration graph needs to be executed on" + " calibration data before convertsion to inference graph"); } auto weight_rmgr = trt_rm->getManager("WeightStore"); TF_CHECK_OK(weight_rmgr->Delete<tensorflow::tensorrt::TRTWeightStore>( @@ -2181,7 +2191,7 @@ tensorflow::Status ConvertCalibrationNodeToEngineNode( return status; } auto trt_engine_node = graph.AddNode(engine_node, &status); - TF_CHECK_OK(status); + TF_RETURN_IF_ERROR(status); for (size_t i = 0; i < out_edges.size(); i++) { VLOG(1) << "Connecting trt_engine_node output " << i << " with " << out_edges.at(i)->dst()->name() << " port " @@ -2279,6 +2289,12 @@ tensorflow::Status InjectCalibrationNode(tensorrt::convert::SubGraphParams& s) { input_dtypes.push_back(tf_dtype); nvinfer1::DataType dtype(nvinfer1::DataType::kFLOAT); + auto type_status = ConvertDType(tf_dtype, &dtype); + if (type_status != tensorflow::Status::OK()) { + LOG(WARNING) << "Data type conversion for input '" << node_name + << "' failed"; + return type_status; + } TF_CHECK_OK(ConvertDType(tf_dtype, &dtype)); VLOG(2) << "accessing output index of: " << output_idx @@ -2346,8 +2362,8 @@ tensorflow::Status InjectCalibrationNode(tensorrt::convert::SubGraphParams& s) { output_names.push_back(tensor_name); auto tensor_or_weights = converter.get_tensor(tensor_name); if (!tensor_or_weights.is_tensor()) { - return tensorflow::errors::InvalidArgument( - "Output node is weights not tensor"); + return tensorflow::errors::InvalidArgument("Output node'" + tensor_name + + "' is weights not tensor"); } nvinfer1::ITensor* tensor = tensor_or_weights.tensor(); if (!tensor) { @@ -2504,7 +2520,11 @@ tensorflow::Status ConvertSubGraphToTensorRTNodeDef( input_dtypes.push_back(tf_dtype); nvinfer1::DataType dtype(nvinfer1::DataType::kFLOAT); - TF_CHECK_OK(ConvertDType(tf_dtype, &dtype)); + auto type_status = ConvertDType(tf_dtype, &dtype); + if (type_status != tensorflow::Status::OK()) { + LOG(WARNING) << "Type conversion failed for " << node_name; + return type_status; + } VLOG(2) << "Accessing output index of: " << output_idx << ", at node: " << node_name @@ -2515,8 +2535,12 @@ tensorflow::Status ConvertSubGraphToTensorRTNodeDef( // TODO(jie): TRT 3.x only support 4 dimensional input tensor. // update the code once TRT 4.0 comes out. - if (op_info.shape().dim_size() != 4) - return tensorflow::errors::Unimplemented("require 4 dimensional input"); + if (op_info.shape().dim_size() != 4) { + string err_str = "Require 4 dimensional input."; + StrAppend(&err_str, " Got ", op_info.shape().dim_size(), " ", + shape_inference_node_name); + return tensorflow::errors::Unimplemented(err_str); + } for (int i = 1; i < op_info.shape().dim_size(); i++) { VLOG(2) << "dimension: " << i @@ -2577,8 +2601,8 @@ tensorflow::Status ConvertSubGraphToTensorRTNodeDef( output_names.push_back(tensor_name); auto tensor_or_weights = converter.get_tensor(tensor_name); if (!tensor_or_weights.is_tensor()) { - return tensorflow::errors::InvalidArgument( - "Output node is weights not tensor"); + return tensorflow::errors::InvalidArgument("Output node '" + tensor_name + + "' is weights not tensor"); } nvinfer1::ITensor* tensor = tensor_or_weights.tensor(); if (!tensor) { @@ -2622,7 +2646,8 @@ tensorflow::Status ConvertSubGraphToTensorRTNodeDef( } TF_RETURN_IF_ERROR(weight_rmgr->Delete<tensorflow::tensorrt::TRTWeightStore>( engine_name, engine_name)); - LOG(INFO) << "finished engine " << engine_name; + LOG(INFO) << "finished engine " << engine_name << " containing " + << s.subgraph_node_ids.size() << " nodes"; // Build the TRT op tensorflow::NodeDefBuilder op_builder(engine_name, "TRTEngineOp"); diff --git a/tensorflow/contrib/tensorrt/segment/segment.cc b/tensorflow/contrib/tensorrt/segment/segment.cc index 6193f0b0a1..8fc4697c51 100644 --- a/tensorflow/contrib/tensorrt/segment/segment.cc +++ b/tensorflow/contrib/tensorrt/segment/segment.cc @@ -80,13 +80,20 @@ void ContractEdge(tensorflow::Edge* edge, tensorflow::Graph* graph, std::vector<const tensorflow::Edge*> in_edges(dst->in_edges().begin(), dst->in_edges().end()); for (const tensorflow::Edge* in_edge : in_edges) { - if (in_edge->src() != src) { - tensorflow::Edge* e = const_cast<tensorflow::Edge*>(in_edge); - if (e->src() == graph->source_node()) { - graph->AddEdge(e->src(), e->src_output(), src, - tensorflow::Graph::kControlSlot); - } else { - graph->AddEdge(e->src(), e->src_output(), src, 0 /* input index */); + if (in_edge->IsControlEdge()) { + if (in_edge->src() != src) { + tensorflow::Edge* e = const_cast<tensorflow::Edge*>(in_edge); + graph->AddControlEdge(e->src(), src); + } + } else { + if (in_edge->src() != src) { + tensorflow::Edge* e = const_cast<tensorflow::Edge*>(in_edge); + if (e->src() == graph->source_node()) { + graph->AddEdge(e->src(), e->src_output(), src, + tensorflow::Graph::kControlSlot); + } else { + graph->AddEdge(e->src(), e->src_output(), src, 0 /* input index */); + } } } } @@ -94,12 +101,19 @@ void ContractEdge(tensorflow::Edge* edge, tensorflow::Graph* graph, std::vector<const tensorflow::Edge*> out_edges(dst->out_edges().begin(), dst->out_edges().end()); for (const tensorflow::Edge* out_edge : out_edges) { - tensorflow::Edge* e = const_cast<tensorflow::Edge*>(out_edge); - if (e->dst() == graph->sink_node()) { - graph->AddEdge(src, tensorflow::Graph::kControlSlot, e->dst(), - e->dst_input()); + if (out_edge->IsControlEdge()) { + tensorflow::Edge* e = const_cast<tensorflow::Edge*>(out_edge); + graph->AddControlEdge(src, e->dst()); } else { - graph->AddEdge(src, 0 /* output index */, e->dst(), e->dst_input()); + tensorflow::Edge* e = const_cast<tensorflow::Edge*>(out_edge); + if (e->dst() == graph->sink_node()) { + VLOG(1) << " edge to sink node " << src->name() << " -> " + << e->dst()->name(); + graph->AddEdge(src, tensorflow::Graph::kControlSlot, e->dst(), + e->dst_input()); + } else { + graph->AddEdge(src, 0 /* output index */, e->dst(), e->dst_input()); + } } } @@ -118,7 +132,7 @@ void ContractEdge(tensorflow::Edge* edge, tensorflow::Graph* graph, tensorflow::Status SegmentGraph( const tensorflow::GraphDef& gdef, - const std::function<bool(const tensorflow::NodeDef&)>& candidate_fn, + const std::function<bool(const tensorflow::Node*)>& candidate_fn, const SegmentOptions& options, SegmentNodesVector* segments) { // Create a Graph representation of the GraphDef. tensorflow::FunctionLibraryDefinition flib(tensorflow::OpRegistry::Global(), @@ -136,7 +150,7 @@ tensorflow::Status SegmentGraph( for (int i = 0; i < graph.num_node_ids(); ++i) { tensorflow::Node* node = graph.FindNodeId(i); if (options.exclude_node_list.count(node->name()) != 0 || - !candidate_fn(node->def())) { + !candidate_fn(node)) { node = nullptr; } node_segments.emplace_back(node); @@ -155,7 +169,7 @@ tensorflow::Status SegmentGraph( for (const tensorflow::Node* node : order) { // All output nodes of 'node' have been visited... - VLOG(2) << "Trying node " << node->name(); + VLOG(2) << "Trying node " << node->name() << " id=" << node->id(); // 'node' must be a TRT candidate... if (node_segments[node->id()].Value() == nullptr) { @@ -169,8 +183,12 @@ tensorflow::Status SegmentGraph( while (true) { std::set<const tensorflow::Edge*> contract_edges; for (const tensorflow::Edge* out_edge : node->out_edges()) { - VLOG(2) << "... out node " << out_edge->dst()->name(); - + VLOG(2) << "... out node " << out_edge->dst()->name() << " ( " + << out_edge->dst()->id() << " <- " << node->id() << " )"; + if (out_edge->IsControlEdge()) { + VLOG(2) << "... ... Control Edge, Skipping"; + continue; + } // Out node must be TRT candidate... if (node_segments[out_edge->dst()->id()].Value() == nullptr) { VLOG(2) << "... ... not a TRT candidate"; @@ -196,7 +214,8 @@ tensorflow::Status SegmentGraph( const tensorflow::Node* src = contract_edge->src(); const tensorflow::Node* dst = contract_edge->dst(); - VLOG(2) << "Merge " << src->name() << " <- " << dst->name(); + VLOG(2) << "Merge " << src->name() << " <- " << dst->name() << " (" + << src->id() << " <- " << dst->id(); node_segments[src->id()].Merge(&node_segments[dst->id()]); // Contracting the edge leaves disconnected graph edges. diff --git a/tensorflow/contrib/tensorrt/segment/segment.h b/tensorflow/contrib/tensorrt/segment/segment.h index ee6e2b3ed2..7e8685f44a 100644 --- a/tensorflow/contrib/tensorrt/segment/segment.h +++ b/tensorflow/contrib/tensorrt/segment/segment.h @@ -20,10 +20,12 @@ limitations under the License. #include <vector> #include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/graph/graph.h" #include "tensorflow/core/lib/core/status.h" #include "tensorflow/core/platform/types.h" namespace tensorflow { + namespace tensorrt { namespace segment { @@ -46,7 +48,7 @@ struct SegmentOptions { // @return the status. tensorflow::Status SegmentGraph( const tensorflow::GraphDef& gdef, - const std::function<bool(const tensorflow::NodeDef&)>& candidate_fn, + const std::function<bool(const tensorflow::Node*)>& candidate_fn, const SegmentOptions& options, SegmentNodesVector* segments); } // namespace segment diff --git a/tensorflow/contrib/tensorrt/segment/segment_test.cc b/tensorflow/contrib/tensorrt/segment/segment_test.cc index 74cbc5f2b3..7ddabec268 100644 --- a/tensorflow/contrib/tensorrt/segment/segment_test.cc +++ b/tensorflow/contrib/tensorrt/segment/segment_test.cc @@ -35,7 +35,7 @@ class SegmentTest : public ::testing::Test { TF_Operation* Add(TF_Operation* l, TF_Operation* r, TF_Graph* graph, TF_Status* s, const char* name); - std::function<bool(const NodeDef&)> MakeCandidateFn( + std::function<bool(const Node*)> MakeCandidateFn( const std::set<string>& node_names); protected: @@ -60,10 +60,10 @@ bool SegmentTest::GetGraphDef(TF_Graph* graph, return ret; } -std::function<bool(const NodeDef&)> SegmentTest::MakeCandidateFn( +std::function<bool(const Node*)> SegmentTest::MakeCandidateFn( const std::set<string>& node_names) { - return [node_names](const NodeDef& node) -> bool { - return node_names.find(node.name()) != node_names.end(); + return [node_names](const Node* node) -> bool { + return node_names.find(node->name()) != node_names.end(); }; } diff --git a/tensorflow/contrib/timeseries/python/timeseries/ar_model.py b/tensorflow/contrib/timeseries/python/timeseries/ar_model.py index ff140efd48..4f6527a546 100644 --- a/tensorflow/contrib/timeseries/python/timeseries/ar_model.py +++ b/tensorflow/contrib/timeseries/python/timeseries/ar_model.py @@ -70,7 +70,7 @@ class ARModel(model.TimeSeriesModel): input_window_size: Number of past time steps of data to look at when doing the regression. output_window_size: Number of future time steps to predict. Note that - setting it to > 1 empiricaly seems to give a better fit. + setting it to > 1 empirically seems to give a better fit. num_features: number of input features per time step. num_time_buckets: Number of buckets into which to divide (time % periodicity) for generating time based features. diff --git a/tensorflow/contrib/timeseries/python/timeseries/math_utils.py b/tensorflow/contrib/timeseries/python/timeseries/math_utils.py index 23452a81c3..26793c80bf 100644 --- a/tensorflow/contrib/timeseries/python/timeseries/math_utils.py +++ b/tensorflow/contrib/timeseries/python/timeseries/math_utils.py @@ -185,7 +185,7 @@ def batch_matrix_pow(matrices, powers): { matmul(A, power(matmul(A, A), (p - 1) / 2)) for odd p power(A, 0) = I - The power(A, 0) = I case is handeled by starting with accumulator set to the + The power(A, 0) = I case is handled by starting with accumulator set to the identity matrix; matrices with zero residual powers are passed through unchanged. diff --git a/tensorflow/contrib/timeseries/python/timeseries/state_space_models/varma.py b/tensorflow/contrib/timeseries/python/timeseries/state_space_models/varma.py index 1afc58cfb2..6746dd7b43 100644 --- a/tensorflow/contrib/timeseries/python/timeseries/state_space_models/varma.py +++ b/tensorflow/contrib/timeseries/python/timeseries/state_space_models/varma.py @@ -107,7 +107,7 @@ class VARMA(state_space_model.StateSpaceModel): Returns: the state transition matrix. It has shape - [self.state_dimendion, self.state_dimension]. + [self.state_dimension, self.state_dimension]. """ # Pad any unused AR blocks with zeros. The extra state is necessary if # ma_order >= ar_order. @@ -127,7 +127,7 @@ class VARMA(state_space_model.StateSpaceModel): Returns: the state noise transform matrix. It has shape - [self.state_dimendion, self.num_features]. + [self.state_dimension, self.num_features]. """ # Noise is broadcast, through the moving average coefficients, to # un-observed parts of the latent state. diff --git a/tensorflow/core/api_def/base_api/api_def_MatrixSolveLs.pbtxt b/tensorflow/core/api_def/base_api/api_def_MatrixSolveLs.pbtxt index 51d91399f8..e667c328ae 100644 --- a/tensorflow/core/api_def/base_api/api_def_MatrixSolveLs.pbtxt +++ b/tensorflow/core/api_def/base_api/api_def_MatrixSolveLs.pbtxt @@ -49,14 +49,14 @@ in the batch: If `fast` is `True`, then the solution is computed by solving the normal equations using Cholesky decomposition. Specifically, if \\(m \ge n\\) then \\(X = (A^H A + \lambda I)^{-1} A^H B\\), which solves the least-squares -problem \\(X = \mathrm{argmin}_{Z \in \Re^{n \times k} } ||A Z - B||_F^2 + -\lambda ||Z||_F^2\\). If \\(m \lt n\\) then `output` is computed as +problem \\(X = \mathrm{argmin}_{Z \in \Re^{n \times k} } ||A Z - B||_F^2 + \lambda ||Z||_F^2\\). +If \\(m \lt n\\) then `output` is computed as \\(X = A^H (A A^H + \lambda I)^{-1} B\\), which (for \\(\lambda = 0\\)) is the minimum-norm solution to the under-determined linear system, i.e. \\(X = \mathrm{argmin}_{Z \in \mathbb{C}^{n \times k} } ||Z||_F^2 \\), subject to \\(A Z = B\\). Notice that the fast path is only numerically stable when \\(A\\) is numerically full rank and has a condition number -\\(\mathrm{cond}(A) \lt \frac{1}{\sqrt{\epsilon_{mach} } }\\) or\\(\lambda\\) is +\\(\mathrm{cond}(A) \lt \frac{1}{\sqrt{\epsilon_{mach} } }\\) or \\(\lambda\\) is sufficiently large. If `fast` is `False` an algorithm based on the numerically robust complete diff --git a/tensorflow/core/common_runtime/mkl_cpu_allocator.cc b/tensorflow/core/common_runtime/mkl_cpu_allocator.cc index 43a909466e..829c19204a 100644 --- a/tensorflow/core/common_runtime/mkl_cpu_allocator.cc +++ b/tensorflow/core/common_runtime/mkl_cpu_allocator.cc @@ -19,9 +19,6 @@ limitations under the License. namespace tensorflow { -constexpr const char* MklCPUAllocator::kMaxLimitStr; -constexpr const size_t MklCPUAllocator::kDefaultMaxLimit; - } // namespace tensorflow #endif // INTEL_MKL diff --git a/tensorflow/core/framework/common_shape_fns.cc b/tensorflow/core/framework/common_shape_fns.cc index 623248b6ce..2fb17c2b02 100644 --- a/tensorflow/core/framework/common_shape_fns.cc +++ b/tensorflow/core/framework/common_shape_fns.cc @@ -1210,7 +1210,7 @@ Status ConcatV2Shape(InferenceContext* c) { c->num_inputs() - 1 /* dim_index */); } -Status BroadcastBinaryOpShapeFn(InferenceContext* c) { +Status BroadcastBinaryOpOutputShapeFn(InferenceContext* c, int output_index) { ShapeHandle shape_x = c->input(0); ShapeHandle shape_y = c->input(1); if (!c->RankKnown(shape_x) || !c->RankKnown(shape_y)) { @@ -1272,7 +1272,7 @@ Status BroadcastBinaryOpShapeFn(InferenceContext* c) { } } - c->set_output(0, c->MakeShape(dims)); + c->set_output(output_index, c->MakeShape(dims)); return Status::OK(); } diff --git a/tensorflow/core/framework/common_shape_fns.h b/tensorflow/core/framework/common_shape_fns.h index 293c40e04d..789746b403 100644 --- a/tensorflow/core/framework/common_shape_fns.h +++ b/tensorflow/core/framework/common_shape_fns.h @@ -265,9 +265,15 @@ Status ConcatShape(shape_inference::InferenceContext* c, // Shape function for concat operations. Status ConcatV2Shape(shape_inference::InferenceContext* c); +// Shape function for binary operators that broadcast their inputs +// and with output to output_index. +Status BroadcastBinaryOpOutputShapeFn(InferenceContext* c, int output_index); + // Shape function for binary operators that broadcast their inputs. // Tested by ops/math_ops_test.cc. -Status BroadcastBinaryOpShapeFn(InferenceContext* c); +inline Status BroadcastBinaryOpShapeFn(InferenceContext* c) { + return BroadcastBinaryOpOutputShapeFn(c, 0); +} // Shape function for random operations. Status RandomShape(shape_inference::InferenceContext* c); diff --git a/tensorflow/core/framework/shape_inference.h b/tensorflow/core/framework/shape_inference.h index e3cc848a16..accc587000 100644 --- a/tensorflow/core/framework/shape_inference.h +++ b/tensorflow/core/framework/shape_inference.h @@ -317,6 +317,7 @@ class InferenceContext { input_tensors_as_shapes_ = input_tensors_as_shapes; } + ShapeHandle output(int64 idx) const { return outputs_[idx]; } void set_output(int idx, ShapeHandle shape) { outputs_[idx] = shape; } Status set_output(StringPiece output_name, const std::vector<ShapeHandle>& shapes); diff --git a/tensorflow/core/kernels/mkl_fused_batch_norm_op.cc b/tensorflow/core/kernels/mkl_fused_batch_norm_op.cc index 333a6570dc..62aafa7930 100644 --- a/tensorflow/core/kernels/mkl_fused_batch_norm_op.cc +++ b/tensorflow/core/kernels/mkl_fused_batch_norm_op.cc @@ -933,7 +933,7 @@ class MklFusedBatchNormOp : public OpKernel { bool is_training_; T* mean_values_; T* variance_values_; - size_t depth_; // batch normalization is done for per channel. + int depth_; // batch normalization is done for per channel. void ExtractParams(OpKernelContext* context) { const Tensor& input = MklGetInput(context, 0); diff --git a/tensorflow/core/kernels/segment_reduction_ops.h b/tensorflow/core/kernels/segment_reduction_ops.h index 4abfbfb1a6..7badc00572 100644 --- a/tensorflow/core/kernels/segment_reduction_ops.h +++ b/tensorflow/core/kernels/segment_reduction_ops.h @@ -23,6 +23,13 @@ limitations under the License. // non-GPU targets. This only breaks in clang, because it's more strict for // template code and CudaAtomicMax is used in template context. +// This file requires the following include because it uses CudaAtomicMax: +// #include "tensorflow/core/util/cuda_kernel_helper.h" + +// Unfortunately we can't add the #include, since it breaks compilation for +// non-GPU targets. This only breaks in clang, because it's more strict for +// template code and CudaAtomicMax is used in template context. + #include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" #include "tensorflow/core/framework/tensor.h" #include "tensorflow/core/framework/tensor_shape.h" diff --git a/tensorflow/core/kernels/snapshot_op.cc b/tensorflow/core/kernels/snapshot_op.cc index 50157d5d48..fe04dcf72e 100644 --- a/tensorflow/core/kernels/snapshot_op.cc +++ b/tensorflow/core/kernels/snapshot_op.cc @@ -22,6 +22,26 @@ limitations under the License. namespace tensorflow { typedef Eigen::ThreadPoolDevice CPUDevice; +typedef Eigen::GpuDevice GPUDevice; + +template <typename Device, typename Scalar> +class SnapshotOp : public OpKernel { + public: + explicit SnapshotOp(OpKernelConstruction* context) : OpKernel(context) {} + + void Compute(OpKernelContext* context) override { + const Tensor& input = context->input(0); + Tensor* output = nullptr; + // Try to use buffer forwarding to avoid an explicit copy. + OP_REQUIRES_OK(context, context->forward_input_or_allocate_output( + {0}, 0, input.shape(), &output)); + if (!output->SharesBufferWith(input)) { + functor::Snapshot<Device, Scalar> functor; + functor(context->eigen_device<Device>(), input.flat<Scalar>(), + output->flat<Scalar>()); + } + } +}; #define REGISTER_KERNEL(TYPE) \ REGISTER_KERNEL_BUILDER( \ @@ -31,6 +51,16 @@ typedef Eigen::ThreadPoolDevice CPUDevice; TF_CALL_POD_TYPES(REGISTER_KERNEL); #undef REGISTER_KERNEL +#if GOOGLE_CUDA +#define REGISTER_KERNEL(TYPE) \ + REGISTER_KERNEL_BUILDER( \ + Name("Snapshot").Device(DEVICE_GPU).TypeConstraint<TYPE>("T"), \ + SnapshotOp<GPUDevice, TYPE>); + +TF_CALL_POD_TYPES(REGISTER_KERNEL); +#undef REGISTER_KERNEL +#endif + #if TENSORFLOW_USE_SYCL typedef Eigen::SyclDevice SyclDevice; #define REGISTER_SYCL_KERNEL(TYPE) \ diff --git a/tensorflow/core/kernels/snapshot_op.h b/tensorflow/core/kernels/snapshot_op.h index b94834f159..a18065d42b 100644 --- a/tensorflow/core/kernels/snapshot_op.h +++ b/tensorflow/core/kernels/snapshot_op.h @@ -26,29 +26,19 @@ limitations under the License. #include "tensorflow/core/framework/op_kernel.h" namespace tensorflow { +namespace functor { +// Functor used by SnapshotOp. template <typename Device, typename Scalar> -class SnapshotOp : public OpKernel { - public: - explicit SnapshotOp(OpKernelConstruction* context) : OpKernel(context) {} - - void Compute(OpKernelContext* context) override { - const Tensor& input = context->input(0); - Tensor* output = nullptr; - // Try to use buffer forwarding to avoid an explicit copy. - OP_REQUIRES_OK(context, context->forward_input_or_allocate_output( - {0}, 0, input.shape(), &output)); - if (!output->SharesBufferWith(input)) { - // We had to allocate a new buffer since the refcount on the input was - // greater than 1. Copy the input to the new buffer. - const Device& device = context->eigen_device<Device>(); - device.memcpy(output->template flat<Scalar>().data(), - input.template flat<Scalar>().data(), - input.NumElements() * sizeof(Scalar)); - } +struct Snapshot { + void operator()(const Device& device, + typename TTypes<Scalar>::ConstTensor input, + typename TTypes<Scalar>::Tensor output) { + device.memcpy(output.data(), input.data(), input.size() * sizeof(Scalar)); } }; +} // namespace functor } // namespace tensorflow #endif // TENSORFLOW_KERNELS_SNAPSHOT_OP_H_ diff --git a/tensorflow/core/kernels/snapshot_op_gpu.cu.cc b/tensorflow/core/kernels/snapshot_op_gpu.cu.cc index 52070be838..e4e3bd5220 100644 --- a/tensorflow/core/kernels/snapshot_op_gpu.cu.cc +++ b/tensorflow/core/kernels/snapshot_op_gpu.cu.cc @@ -24,13 +24,10 @@ limitations under the License. namespace tensorflow { typedef Eigen::GpuDevice GPUDevice; -#define REGISTER_KERNEL(TYPE) \ - REGISTER_KERNEL_BUILDER( \ - Name("Snapshot").Device(DEVICE_GPU).TypeConstraint<TYPE>("T"), \ - SnapshotOp<GPUDevice, TYPE>); +// Definition of the GPU implementations declared in softsign_op.cc. +#define DEFINE_GPU_KERNELS(T) template struct functor::Snapshot<GPUDevice, T>; -TF_CALL_POD_TYPES(REGISTER_KERNEL); -#undef REGISTER_KERNEL +TF_CALL_POD_TYPES(DEFINE_GPU_KERNELS); } // namespace tensorflow diff --git a/tensorflow/core/kernels/xent_op.cc b/tensorflow/core/kernels/xent_op.cc index a6a71fdfaf..9a3612bd72 100644 --- a/tensorflow/core/kernels/xent_op.cc +++ b/tensorflow/core/kernels/xent_op.cc @@ -17,12 +17,14 @@ limitations under the License. #define EIGEN_USE_THREADS -#include "tensorflow/core/kernels/xent_op.h" #include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + #include "tensorflow/core/framework/op_kernel.h" #include "tensorflow/core/framework/register_types.h" #include "tensorflow/core/framework/tensor.h" #include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/kernels/xent_op.h" +#include "tensorflow/core/util/bcast.h" namespace tensorflow { @@ -41,37 +43,56 @@ class SoftmaxXentWithLogitsOp : public OpKernel { void Compute(OpKernelContext* context) override { const Tensor& logits_in = context->input(0); const Tensor& labels_in = context->input(1); - OP_REQUIRES(context, logits_in.IsSameSize(labels_in), - errors::InvalidArgument( - "logits and labels must be same size: logits_size=", - logits_in.shape().DebugString(), - " labels_size=", labels_in.shape().DebugString())); - OP_REQUIRES(context, TensorShapeUtils::IsMatrix(logits_in.shape()), - errors::InvalidArgument("logits must be 2-dimensional")); - // As we already tested that both inputs have the same shape no need to - // check that "labels" is a matrix too. + + TensorShape shape_in = logits_in.shape(); + + BCast bcast(BCast::FromShape(logits_in.shape()), + BCast::FromShape(labels_in.shape())); + if (!logits_in.IsSameSize(labels_in)) { + OP_REQUIRES(context, bcast.IsValid(), + errors::InvalidArgument( + "logits and labels must be broadcastable: logits_size=", + logits_in.shape().DebugString(), + " labels_size=", labels_in.shape().DebugString())); + shape_in = BCast::ToShape(bcast.output_shape()); + } + OP_REQUIRES(context, TensorShapeUtils::IsMatrix(shape_in), + errors::InvalidArgument("logits and labels must be beither " + "2-dimensional, or roadcasted to " + "2-dimensional")); // loss is 1-D (one per example), and size is batch_size. Tensor scratch; OP_REQUIRES_OK( context, context->allocate_temp(DataTypeToEnum<T>::value, - TensorShape({logits_in.dim_size(0), 1}), + TensorShape({shape_in.dim_size(0), 1}), &scratch)); Tensor* loss_out = nullptr; OP_REQUIRES_OK(context, context->allocate_output( - 0, TensorShape({logits_in.dim_size(0)}), &loss_out)); + 0, TensorShape({shape_in.dim_size(0)}), &loss_out)); Tensor* back_out = nullptr; // Try to reuse the logits_in buffer for the backprop output. OP_REQUIRES_OK(context, context->forward_input_or_allocate_output( - {0}, 1, logits_in.shape(), &back_out)); - if (logits_in.dim_size(0) > 0) { + {0}, 1, shape_in, &back_out)); + if (shape_in.dim_size(0) > 0) { functor::XentFunctor<Device, T> functor; - functor(context->eigen_device<Device>(), logits_in.matrix<T>(), - labels_in.matrix<T>(), scratch.matrix<T>(), loss_out->vec<T>(), - back_out->matrix<T>()); + if (logits_in.IsSameSize(labels_in)) { + functor(context->eigen_device<Device>(), shape_in.AsEigenDSizes<2>(), + Eigen::array<Eigen::DenseIndex, 2>{1, 1}, + Eigen::array<Eigen::DenseIndex, 2>{1, 1}, logits_in.matrix<T>(), + labels_in.matrix<T>(), scratch.matrix<T>(), loss_out->vec<T>(), + back_out->matrix<T>()); + } else { + functor(context->eigen_device<Device>(), shape_in.AsEigenDSizes<2>(), + BCast::ToIndexArray<2>(bcast.x_bcast()), + BCast::ToIndexArray<2>(bcast.y_bcast()), + logits_in.template shaped<T, 2>(bcast.x_reshape()), + labels_in.template shaped<T, 2>(bcast.y_reshape()), + scratch.matrix<T>(), loss_out->vec<T>(), back_out->matrix<T>()); + } } } }; @@ -81,13 +102,17 @@ class SoftmaxXentWithLogitsOp : public OpKernel { namespace functor { template <typename Device, typename T> struct XentFunctorBase { - void operator()(const Device& d, typename TTypes<T>::ConstMatrix logits, + void operator()(const Device& d, + const Eigen::DSizes<Eigen::DenseIndex, 2>& shape, + const Eigen::array<Eigen::DenseIndex, 2>& logits_bcast, + const Eigen::array<Eigen::DenseIndex, 2>& labels_bcast, + typename TTypes<T>::ConstMatrix logits, typename TTypes<T>::ConstMatrix labels, typename TTypes<T>::Matrix scratch, typename TTypes<T>::Vec loss, typename TTypes<T>::Matrix backprop) { - XentEigenImpl<Device, T>::Compute(d, logits, labels, scratch, loss, - backprop); + XentEigenImpl<Device, T>::Compute(d, shape, logits_bcast, labels_bcast, + logits, labels, scratch, loss, backprop); } }; diff --git a/tensorflow/core/kernels/xent_op.h b/tensorflow/core/kernels/xent_op.h index e689fca7ff..87be17fca9 100644 --- a/tensorflow/core/kernels/xent_op.h +++ b/tensorflow/core/kernels/xent_op.h @@ -18,6 +18,7 @@ limitations under the License. // Functor definition for XentOp, must be compilable by nvcc. #include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + #include "tensorflow/core/framework/tensor_types.h" namespace tensorflow { @@ -33,7 +34,11 @@ struct XentFunctor { // scratch: temporary tensor, dims: batch_size, 1 // loss: output tensor for the loss, dims: batch_size. // backprop: output tensor for the backprop, dims: batch_size, num_classes. - void operator()(const Device& d, typename TTypes<T>::ConstMatrix logits, + void operator()(const Device &d, + const Eigen::DSizes<Eigen::DenseIndex, 2> &shape, + const Eigen::array<Eigen::DenseIndex, 2> &logits_bcast, + const Eigen::array<Eigen::DenseIndex, 2> &labels_bcast, + typename TTypes<T>::ConstMatrix logits, typename TTypes<T>::ConstMatrix labels, typename TTypes<T>::Matrix scratch, typename TTypes<T>::Vec loss, @@ -45,7 +50,11 @@ struct XentFunctor { // specializations for both device types. template <typename Device, typename T> struct XentEigenImpl { - static void Compute(const Device& d, typename TTypes<T>::ConstMatrix logits, + static void Compute(const Device &d, + const Eigen::DSizes<Eigen::DenseIndex, 2> &shape, + const Eigen::array<Eigen::DenseIndex, 2> &logits_bcast, + const Eigen::array<Eigen::DenseIndex, 2> &labels_bcast, + typename TTypes<T>::ConstMatrix logits, typename TTypes<T>::ConstMatrix labels, typename TTypes<T>::Matrix scratch, typename TTypes<T>::Vec loss, @@ -57,8 +66,8 @@ struct XentEigenImpl { const int kBatchDim = 0; const int kClassDim = 1; - const int batch_size = logits.dimension(kBatchDim); - const int num_classes = logits.dimension(kClassDim); + const int batch_size = shape[kBatchDim]; + const int num_classes = shape[kClassDim]; // These arrays are used to reduce along the class dimension, and broadcast // the resulting value to all classes. @@ -84,10 +93,12 @@ struct XentEigenImpl { #endif // max_logits along classes. - scratch.reshape(batch_only).device(d) = logits.maximum(along_class); + scratch.reshape(batch_only).device(d) = + logits.broadcast(logits_bcast).maximum(along_class); // logits - max_logits. - backprop.device(d) = logits - scratch.broadcast(one_by_class); + backprop.device(d) = + logits.broadcast(logits_bcast) - scratch.broadcast(one_by_class); // sum(exp(logits - max_logits)) along classes. scratch.reshape(batch_only).device(d) = backprop.exp().sum(along_class); @@ -99,15 +110,15 @@ struct XentEigenImpl { // sum(-labels * // ((logits - max_logits) - log(sum(exp(logits - max_logits))))) // along classes - loss.device(d) = - (labels * (scratch.log().eval().broadcast(one_by_class) - backprop)) - .eval() - .sum(along_class); + loss.device(d) = (labels.broadcast(labels_bcast) * + (scratch.log().eval().broadcast(one_by_class) - backprop)) + .eval() + .sum(along_class); // backprop: prob - labels, where // prob = exp(logits - max_logits) / sum(exp(logits - max_logits)) - backprop.device(d) = - (backprop.exp() / scratch.broadcast(one_by_class)) - labels; + backprop.device(d) = (backprop.exp() / scratch.broadcast(one_by_class)) - + labels.broadcast(labels_bcast); } }; diff --git a/tensorflow/core/kernels/xent_op_gpu.cu.cc b/tensorflow/core/kernels/xent_op_gpu.cu.cc index 05ee7da490..2c0c0b3a02 100644 --- a/tensorflow/core/kernels/xent_op_gpu.cu.cc +++ b/tensorflow/core/kernels/xent_op_gpu.cu.cc @@ -31,12 +31,17 @@ typedef Eigen::GpuDevice GPUDevice; namespace functor { template <typename T> struct XentFunctor<GPUDevice, T> { - void operator()(const GPUDevice& d, typename TTypes<T>::ConstMatrix logits, + void operator()(const GPUDevice &d, + const Eigen::DSizes<Eigen::DenseIndex, 2> &shape, + const Eigen::array<Eigen::DenseIndex, 2> &logits_bcast, + const Eigen::array<Eigen::DenseIndex, 2> &labels_bcast, + typename TTypes<T>::ConstMatrix logits, typename TTypes<T>::ConstMatrix labels, typename TTypes<T>::Matrix scratch, typename TTypes<T>::Vec loss, typename TTypes<T>::Matrix backprop) { - XentEigenImpl<GPUDevice, T>::Compute(d, logits, labels, scratch, loss, + XentEigenImpl<GPUDevice, T>::Compute(d, shape, logits_bcast, labels_bcast, + logits, labels, scratch, loss, backprop); } }; diff --git a/tensorflow/core/ops/array_ops.cc b/tensorflow/core/ops/array_ops.cc index 88d2aa3f41..111670c361 100644 --- a/tensorflow/core/ops/array_ops.cc +++ b/tensorflow/core/ops/array_ops.cc @@ -794,11 +794,35 @@ REGISTER_OP("ReverseV2") ShapeHandle input = c->input(0); ShapeHandle axis; TF_RETURN_IF_ERROR(c->WithRank(c->input(1), 1, &axis)); - // TODO(aselle): if input(0)'s dimension is known we could validate axis if (c->Rank(input) > 8) { return errors::InvalidArgument( "reverse does not work on tensors with more than 8 dimensions"); } + const Tensor* axis_tensor = c->input_tensor(1); + if (axis_tensor != nullptr && c->RankKnown(input)) { + int32 rank = c->Rank(input); + std::vector<int64> axis_value; + if (axis_tensor->dtype() == DT_INT32) { + axis_value = AsInt64<int32>(axis_tensor, axis_tensor->NumElements()); + } else { + axis_value = AsInt64<int64>(axis_tensor, axis_tensor->NumElements()); + } + std::vector<bool> axes_dense(c->Rank(input), false); + for (int i = 0; i < axis_value.size(); i++) { + int64 canonical_axis = + axis_value[i] < 0 ? rank + axis_value[i] : axis_value[i]; + if (canonical_axis < 0 || canonical_axis >= rank) { + return errors::InvalidArgument("'axis'[", i, "] = ", axis_value[i], + " is out of valid range [", 0, ", ", + rank - 1); + } + if (axes_dense[canonical_axis]) { + return errors::InvalidArgument("axis ", canonical_axis, + " specified more than once."); + } + axes_dense[canonical_axis] = true; + } + } c->set_output(0, input); return Status::OK(); }); diff --git a/tensorflow/core/ops/nn_ops.cc b/tensorflow/core/ops/nn_ops.cc index 1f4e9753c3..6c2fc60bab 100644 --- a/tensorflow/core/ops/nn_ops.cc +++ b/tensorflow/core/ops/nn_ops.cc @@ -1062,12 +1062,27 @@ REGISTER_OP("SoftmaxCrossEntropyWithLogits") .Attr("T: {half, bfloat16, float, double}") .SetShapeFn([](InferenceContext* c) { ShapeHandle input; - TF_RETURN_IF_ERROR(c->WithRank(c->input(0), 2, &input)); - TF_RETURN_IF_ERROR(c->Merge(input, c->input(1), &input)); + if (c->WithRank(c->input(0), 2, &input) == Status::OK() && + c->Merge(input, c->input(1), &input) == Status::OK()) { + DimensionHandle batch_size = c->Dim(input, 0); + c->set_output(0, c->Vector(batch_size)); + c->set_output(1, input); + return Status::OK(); + } + TF_RETURN_IF_ERROR(BroadcastBinaryOpOutputShapeFn(c, 1)); - DimensionHandle batch_size = c->Dim(input, 0); + if (!c->RankKnown(c->output(1))) { + return errors::InvalidArgument( + "Shape must be broadcasted with rank 2, but is rank is unknown."); + } + + if (c->Rank(c->output(1)) != 2) { + return errors::InvalidArgument( + "Shape must be broadcasted with rank 2, but is rank ", + c->Rank(c->output(1))); + } + DimensionHandle batch_size = c->Dim(c->output(1), 0); c->set_output(0, c->Vector(batch_size)); - c->set_output(1, input); return Status::OK(); }); diff --git a/tensorflow/core/ops/nn_ops_test.cc b/tensorflow/core/ops/nn_ops_test.cc index 1b17a7cda6..289b953055 100644 --- a/tensorflow/core/ops/nn_ops_test.cc +++ b/tensorflow/core/ops/nn_ops_test.cc @@ -410,10 +410,18 @@ TEST(NNOpsTest, SoftmaxCrossEntropyWithLogits_ShapeFn) { INFER_OK(op, "[1,?];[?,2]", "[d0_0];[d0_0,d0_1|d1_1]"); INFER_OK(op, "[?,2];[1,2]", "[d1_0];in1"); - INFER_ERROR("Dimension 0 in both shapes must be equal, but are 1 and 2", op, - "[1,?];[2,?]"); - INFER_ERROR("Shape must be rank 2 but is rank 3", op, "[1,2,3];?"); - INFER_ERROR("Shapes must be equal rank, but are 2 and 3", op, "?;[1,2,3]"); + INFER_ERROR("Shape must be broadcasted with rank 2", op, "[1,2,3];?"); + INFER_ERROR("Shape must be broadcasted with rank 2", op, "?;[1,2,3]"); + + // Broadcast example + // [1,4] and [2,4] are broadcasted to [2,4] + INFER_OK(op, "[1,4];[2,4]", "[d1_0];[d1_0,d0_1|d1_1]"); + // [2,4] and [2,1] are broadcasted to [2,4] + INFER_OK(op, "[2,4];[2,1]", "[d0_0];[d0_0|d1_0,d0_1]"); + // [1,?] and [2,4] are broadcasted to [2,4] + INFER_OK(op, "[1,?];[2,4]", "[d1_0];[d1_0,d0_1|d1_1]"); + // [2,4] and [?,1] are broadcasted to [2,4] + INFER_OK(op, "[2,4];[?,1]", "[d0_0];[d0_0|d1_0,d0_1]"); } TEST(NNOpsTest, SparseSoftmaxCrossEntropyWithLogits_ShapeFn) { diff --git a/tensorflow/core/public/version.h b/tensorflow/core/public/version.h index 22f2c02b78..40eebd1db0 100644 --- a/tensorflow/core/public/version.h +++ b/tensorflow/core/public/version.h @@ -19,12 +19,12 @@ limitations under the License. // TensorFlow uses semantic versioning, see http://semver.org/. #define TF_MAJOR_VERSION 1 -#define TF_MINOR_VERSION 6 +#define TF_MINOR_VERSION 7 #define TF_PATCH_VERSION 0 // TF_VERSION_SUFFIX is non-empty for pre-releases (e.g. "-alpha", "-alpha.1", // "-beta", "-rc", "-rc.1") -#define TF_VERSION_SUFFIX "" +#define TF_VERSION_SUFFIX "-rc1" #define TF_STR_HELPER(x) #x #define TF_STR(x) TF_STR_HELPER(x) diff --git a/tensorflow/docs_src/api_guides/python/contrib.bayesflow.monte_carlo.md b/tensorflow/docs_src/api_guides/python/contrib.bayesflow.monte_carlo.md index 956dccb64f..f3db5857ae 100644 --- a/tensorflow/docs_src/api_guides/python/contrib.bayesflow.monte_carlo.md +++ b/tensorflow/docs_src/api_guides/python/contrib.bayesflow.monte_carlo.md @@ -6,42 +6,42 @@ Monte Carlo integration and helpers. ## Background Monte Carlo integration refers to the practice of estimating an expectation with -a sample mean. For example, given random variable `Z in R^k` with density `p`, +a sample mean. For example, given random variable `Z in \\(R^k\\)` with density `p`, the expectation of function `f` can be approximated like: ``` -E_p[f(Z)] = \int f(z) p(z) dz - ~ S_n - := n^{-1} \sum_{i=1}^n f(z_i), z_i iid samples from p. +$$E_p[f(Z)] = \int f(z) p(z) dz$$ +$$ ~ S_n + := n^{-1} \sum_{i=1}^n f(z_i), z_i\ iid\ samples\ from\ p.$$ ``` -If `E_p[|f(Z)|] < infinity`, then `S_n --> E_p[f(Z)]` by the strong law of large -numbers. If `E_p[f(Z)^2] < infinity`, then `S_n` is asymptotically normal with -variance `Var[f(Z)] / n`. +If `\\(E_p[|f(Z)|] < infinity\\)`, then `\\(S_n\\) --> \\(E_p[f(Z)]\\)` by the strong law of large +numbers. If `\\(E_p[f(Z)^2] < infinity\\)`, then `\\(S_n\\)` is asymptotically normal with +variance `\\(Var[f(Z)] / n\\)`. Practitioners of Bayesian statistics often find themselves wanting to estimate -`E_p[f(Z)]` when the distribution `p` is known only up to a constant. For +`\\(E_p[f(Z)]\\)` when the distribution `p` is known only up to a constant. For example, the joint distribution `p(z, x)` may be known, but the evidence -`p(x) = \int p(z, x) dz` may be intractable. In that case, a parameterized -distribution family `q_lambda(z)` may be chosen, and the optimal `lambda` is the -one minimizing the KL divergence between `q_lambda(z)` and -`p(z | x)`. We only know `p(z, x)`, but that is sufficient to find `lambda`. +`\\(p(x) = \int p(z, x) dz\\)` may be intractable. In that case, a parameterized +distribution family `\\(q_\lambda(z)\\)` may be chosen, and the optimal `\\(\lambda\\)` is the +one minimizing the KL divergence between `\\(q_\lambda(z)\\)` and +`\\(p(z | x)\\)`. We only know `p(z, x)`, but that is sufficient to find `\\(\lambda\\)`. ## Log-space evaluation and subtracting the maximum Care must be taken when the random variable lives in a high dimensional space. -For example, the naive importance sample estimate `E_q[f(Z) p(Z) / q(Z)]` -involves the ratio of two terms `p(Z) / q(Z)`, each of which must have tails -dropping off faster than `O(|z|^{-(k + 1)})` in order to have finite integral. +For example, the naive importance sample estimate `\\(E_q[f(Z) p(Z) / q(Z)]\\)` +involves the ratio of two terms `\\(p(Z) / q(Z)\\)`, each of which must have tails +dropping off faster than `\\(O(|z|^{-(k + 1)})\\)` in order to have finite integral. This ratio would often be zero or infinity up to numerical precision. For that reason, we write ``` -Log E_q[ f(Z) p(Z) / q(Z) ] - = Log E_q[ exp{Log[f(Z)] + Log[p(Z)] - Log[q(Z)] - C} ] + C, where -C := Max[ Log[f(Z)] + Log[p(Z)] - Log[q(Z)] ]. +$$Log E_q[ f(Z) p(Z) / q(Z) ]$$ +$$ = Log E_q[ \exp\{Log[f(Z)] + Log[p(Z)] - Log[q(Z)] - C\} ] + C,$$ where +$$C := Max[ Log[f(Z)] + Log[p(Z)] - Log[q(Z)] ].$$ ``` The maximum value of the exponentiated term will be 0.0, and the expectation diff --git a/tensorflow/docs_src/api_guides/python/contrib.losses.md b/tensorflow/docs_src/api_guides/python/contrib.losses.md index d7f862625e..8b7442216c 100644 --- a/tensorflow/docs_src/api_guides/python/contrib.losses.md +++ b/tensorflow/docs_src/api_guides/python/contrib.losses.md @@ -107,19 +107,19 @@ weighted average over the individual prediction errors: loss = tf.contrib.losses.mean_squared_error(predictions, depths, weight) ``` -@{tf.contrib.losses.absolute_difference} -@{tf.contrib.losses.add_loss} -@{tf.contrib.losses.hinge_loss} -@{tf.contrib.losses.compute_weighted_loss} -@{tf.contrib.losses.cosine_distance} -@{tf.contrib.losses.get_losses} -@{tf.contrib.losses.get_regularization_losses} -@{tf.contrib.losses.get_total_loss} -@{tf.contrib.losses.log_loss} -@{tf.contrib.losses.mean_pairwise_squared_error} -@{tf.contrib.losses.mean_squared_error} -@{tf.contrib.losses.sigmoid_cross_entropy} -@{tf.contrib.losses.softmax_cross_entropy} -@{tf.contrib.losses.sparse_softmax_cross_entropy} +* @{tf.contrib.losses.absolute_difference} +* @{tf.contrib.losses.add_loss} +* @{tf.contrib.losses.hinge_loss} +* @{tf.contrib.losses.compute_weighted_loss} +* @{tf.contrib.losses.cosine_distance} +* @{tf.contrib.losses.get_losses} +* @{tf.contrib.losses.get_regularization_losses} +* @{tf.contrib.losses.get_total_loss} +* @{tf.contrib.losses.log_loss} +* @{tf.contrib.losses.mean_pairwise_squared_error} +* @{tf.contrib.losses.mean_squared_error} +* @{tf.contrib.losses.sigmoid_cross_entropy} +* @{tf.contrib.losses.softmax_cross_entropy} +* @{tf.contrib.losses.sparse_softmax_cross_entropy} diff --git a/tensorflow/docs_src/community/documentation.md b/tensorflow/docs_src/community/documentation.md index 003e0a25ec..6f2107ef40 100644 --- a/tensorflow/docs_src/community/documentation.md +++ b/tensorflow/docs_src/community/documentation.md @@ -477,31 +477,29 @@ should use Markdown in the docstring. Here's a simple example: -```python -def foo(x, y, name="bar"): - """Computes foo. + def foo(x, y, name="bar"): + """Computes foo. - Given two 1-D tensors `x` and `y`, this operation computes the foo. + Given two 1-D tensors `x` and `y`, this operation computes the foo. - Example: + Example: - ``` - # x is [1, 1] - # y is [2, 2] - tf.foo(x, y) ==> [3, 3] - ``` - Args: - x: A `Tensor` of type `int32`. - y: A `Tensor` of type `int32`. - name: A name for the operation (optional). + ``` + # x is [1, 1] + # y is [2, 2] + tf.foo(x, y) ==> [3, 3] + ``` + Args: + x: A `Tensor` of type `int32`. + y: A `Tensor` of type `int32`. + name: A name for the operation (optional). - Returns: - A `Tensor` of type `int32` that is the foo of `x` and `y`. + Returns: + A `Tensor` of type `int32` that is the foo of `x` and `y`. - Raises: - ValueError: If `x` or `y` are not of type `int32`. - """ -``` + Raises: + ValueError: If `x` or `y` are not of type `int32`. + """ ## Description of the docstring sections diff --git a/tensorflow/docs_src/install/install_c.md b/tensorflow/docs_src/install/install_c.md index 0481c97885..9059b3f3b6 100644 --- a/tensorflow/docs_src/install/install_c.md +++ b/tensorflow/docs_src/install/install_c.md @@ -38,7 +38,7 @@ enable TensorFlow for C: OS="linux" # Change to "darwin" for macOS TARGET_DIRECTORY="/usr/local" curl -L \ - "https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-${TF_TYPE}-${OS}-x86_64-1.6.0.tar.gz" | + "https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-${TF_TYPE}-${OS}-x86_64-1.7.0-rc1.tar.gz" | sudo tar -C $TARGET_DIRECTORY -xz The `tar` command extracts the TensorFlow C library into the `lib` diff --git a/tensorflow/docs_src/install/install_go.md b/tensorflow/docs_src/install/install_go.md index 8f89898c92..2e47a6d212 100644 --- a/tensorflow/docs_src/install/install_go.md +++ b/tensorflow/docs_src/install/install_go.md @@ -38,7 +38,7 @@ steps to install this library and enable TensorFlow for Go: TF_TYPE="cpu" # Change to "gpu" for GPU support TARGET_DIRECTORY='/usr/local' curl -L \ - "https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-${TF_TYPE}-$(go env GOOS)-x86_64-1.6.0.tar.gz" | + "https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-${TF_TYPE}-$(go env GOOS)-x86_64-1.7.0-rc1.tar.gz" | sudo tar -C $TARGET_DIRECTORY -xz The `tar` command extracts the TensorFlow C library into the `lib` diff --git a/tensorflow/docs_src/install/install_java.md b/tensorflow/docs_src/install/install_java.md index 0ee9c849e1..eff066d200 100644 --- a/tensorflow/docs_src/install/install_java.md +++ b/tensorflow/docs_src/install/install_java.md @@ -36,7 +36,7 @@ following to the project's `pom.xml` to use the TensorFlow Java APIs: <dependency> <groupId>org.tensorflow</groupId> <artifactId>tensorflow</artifactId> - <version>1.6.0</version> + <version>1.7.0-rc1</version> </dependency> ``` @@ -65,7 +65,7 @@ As an example, these steps will create a Maven project that uses TensorFlow: <dependency> <groupId>org.tensorflow</groupId> <artifactId>tensorflow</artifactId> - <version>1.6.0</version> + <version>1.7.0-rc1</version> </dependency> </dependencies> </project> @@ -123,12 +123,12 @@ instead: <dependency> <groupId>org.tensorflow</groupId> <artifactId>libtensorflow</artifactId> - <version>1.6.0</version> + <version>1.7.0-rc1</version> </dependency> <dependency> <groupId>org.tensorflow</groupId> <artifactId>libtensorflow_jni_gpu</artifactId> - <version>1.6.0</version> + <version>1.7.0-rc1</version> </dependency> ``` @@ -147,7 +147,7 @@ refer to the simpler instructions above instead. Take the following steps to install TensorFlow for Java on Linux or macOS: 1. Download - [libtensorflow.jar](https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-1.6.0.jar), + [libtensorflow.jar](https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-1.7.0-rc1.jar), which is the TensorFlow Java Archive (JAR). 2. Decide whether you will run TensorFlow for Java on CPU(s) only or with @@ -166,7 +166,7 @@ Take the following steps to install TensorFlow for Java on Linux or macOS: OS=$(uname -s | tr '[:upper:]' '[:lower:]') mkdir -p ./jni curl -L \ - "https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow_jni-${TF_TYPE}-${OS}-x86_64-1.6.0.tar.gz" | + "https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow_jni-${TF_TYPE}-${OS}-x86_64-1.7.0-rc1.tar.gz" | tar -xz -C ./jni ### Install on Windows @@ -174,10 +174,10 @@ Take the following steps to install TensorFlow for Java on Linux or macOS: Take the following steps to install TensorFlow for Java on Windows: 1. Download - [libtensorflow.jar](https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-1.6.0.jar), + [libtensorflow.jar](https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-1.7.0-rc1.jar), which is the TensorFlow Java Archive (JAR). 2. Download the following Java Native Interface (JNI) file appropriate for - [TensorFlow for Java on Windows](https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow_jni-cpu-windows-x86_64-1.6.0.zip). + [TensorFlow for Java on Windows](https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow_jni-cpu-windows-x86_64-1.7.0-rc1.zip). 3. Extract this .zip file. @@ -225,7 +225,7 @@ must be part of your `classpath`. For example, you can include the downloaded `.jar` in your `classpath` by using the `-cp` compilation flag as follows: -<pre><b>javac -cp libtensorflow-1.6.0.jar HelloTF.java</b></pre> +<pre><b>javac -cp libtensorflow-1.7.0-rc1.jar HelloTF.java</b></pre> ### Running @@ -239,11 +239,11 @@ two files are available to the JVM: For example, the following command line executes the `HelloTF` program on Linux and macOS X: -<pre><b>java -cp libtensorflow-1.6.0.jar:. -Djava.library.path=./jni HelloTF</b></pre> +<pre><b>java -cp libtensorflow-1.7.0-rc1.jar:. -Djava.library.path=./jni HelloTF</b></pre> And the following command line executes the `HelloTF` program on Windows: -<pre><b>java -cp libtensorflow-1.6.0.jar;. -Djava.library.path=jni HelloTF</b></pre> +<pre><b>java -cp libtensorflow-1.7.0-rc1.jar;. -Djava.library.path=jni HelloTF</b></pre> If the program prints <tt>Hello from <i>version</i></tt>, you've successfully installed TensorFlow for Java and are ready to use the API. If the program diff --git a/tensorflow/docs_src/install/install_linux.md b/tensorflow/docs_src/install/install_linux.md index 5e9a84bff6..27b696696d 100644 --- a/tensorflow/docs_src/install/install_linux.md +++ b/tensorflow/docs_src/install/install_linux.md @@ -165,7 +165,7 @@ Take the following steps to install TensorFlow with Virtualenv: Virtualenv environment: <pre>(tensorflow)$ <b>pip3 install --upgrade \ - https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.6.0-cp34-cp34m-linux_x86_64.whl</b></pre> + https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.7.0rc1-cp34-cp34m-linux_x86_64.whl</b></pre> If you encounter installation problems, see [Common Installation Problems](#common_installation_problems). @@ -270,7 +270,7 @@ take the following steps: <pre> $ <b>sudo pip3 install --upgrade \ - https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.6.0-cp34-cp34m-linux_x86_64.whl</b> + https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.7.0rc1-cp34-cp34m-linux_x86_64.whl</b> </pre> If this step fails, see @@ -456,7 +456,7 @@ Take the following steps to install TensorFlow in an Anaconda environment: <pre> (tensorflow)$ <b>pip install --ignore-installed --upgrade \ - https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.6.0-cp34-cp34m-linux_x86_64.whl</b></pre> + https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.7.0rc1-cp34-cp34m-linux_x86_64.whl</b></pre> <a name="ValidateYourInstallation"></a> ## Validate your installation @@ -630,14 +630,14 @@ This section documents the relevant values for Linux installations. CPU only: <pre> -https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.6.0-cp27-none-linux_x86_64.whl +https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.7.0rc1-cp27-none-linux_x86_64.whl </pre> GPU support: <pre> -https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.6.0-cp27-none-linux_x86_64.whl +https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.7.0rc1-cp27-none-linux_x86_64.whl </pre> Note that GPU support requires the NVIDIA hardware and software described in @@ -649,14 +649,14 @@ Note that GPU support requires the NVIDIA hardware and software described in CPU only: <pre> -https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.6.0-cp34-cp34m-linux_x86_64.whl +https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.7.0rc1-cp34-cp34m-linux_x86_64.whl </pre> GPU support: <pre> -https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.6.0-cp34-cp34m-linux_x86_64.whl +https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.7.0rc1-cp34-cp34m-linux_x86_64.whl </pre> Note that GPU support requires the NVIDIA hardware and software described in @@ -668,14 +668,14 @@ Note that GPU support requires the NVIDIA hardware and software described in CPU only: <pre> -https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.6.0-cp35-cp35m-linux_x86_64.whl +https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.7.0rc1-cp35-cp35m-linux_x86_64.whl </pre> GPU support: <pre> -https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.6.0-cp35-cp35m-linux_x86_64.whl +https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.7.0rc1-cp35-cp35m-linux_x86_64.whl </pre> @@ -687,14 +687,14 @@ Note that GPU support requires the NVIDIA hardware and software described in CPU only: <pre> -https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.6.0-cp36-cp36m-linux_x86_64.whl +https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.7.0rc1-cp36-cp36m-linux_x86_64.whl </pre> GPU support: <pre> -https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.6.0-cp36-cp36m-linux_x86_64.whl +https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.7.0rc1-cp36-cp36m-linux_x86_64.whl </pre> diff --git a/tensorflow/docs_src/install/install_mac.md b/tensorflow/docs_src/install/install_mac.md index 55b460e189..7060ef43da 100644 --- a/tensorflow/docs_src/install/install_mac.md +++ b/tensorflow/docs_src/install/install_mac.md @@ -118,8 +118,8 @@ Take the following steps to install TensorFlow with Virtualenv: Python 2.7, the command to install TensorFlow in the active Virtualenv is as follows: - <pre> $ <b>pip install --upgrade \ - https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.6.0-py2-none-any.whl</b></pre> + <pre> $ <b>pip3 install --upgrade \ + https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.7.0rc1-py3-none-any.whl</b></pre> If you encounter installation problems, see [Common Installation Problems](#common-installation-problems). @@ -241,8 +241,8 @@ take the following steps: you are installing TensorFlow for macOS and Python 2.7 issue the following command: - <pre> $ <b>sudo pip install --upgrade \ - https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.6.0-py2-none-any.whl</b> </pre> + <pre> $ <b>sudo pip3 install --upgrade \ + https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.7.0rc1-py3-none-any.whl</b> </pre> If the preceding command fails, see [installation problems](#common-installation-problems). @@ -350,7 +350,7 @@ Take the following steps to install TensorFlow in an Anaconda environment: TensorFlow for Python 2.7: <pre> (<i>targetDirectory</i>)$ <b>pip install --ignore-installed --upgrade \ - https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.6.0-py2-none-any.whl</b></pre> + https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.7.0rc1-py2-none-any.whl</b></pre> <a name="ValidateYourInstallation"></a> @@ -524,7 +524,7 @@ The value you specify depends on your Python version. <pre> -https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.6.0-py2-none-any.whl +https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.7.0rc1-py2-none-any.whl </pre> @@ -532,5 +532,5 @@ https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.6.0-py2-none-any. <pre> -https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.6.0-py3-none-any.whl +https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.7.0rc1-py3-none-any.whl </pre> diff --git a/tensorflow/docs_src/install/install_sources.md b/tensorflow/docs_src/install/install_sources.md index a7f33819b4..148f80efe2 100644 --- a/tensorflow/docs_src/install/install_sources.md +++ b/tensorflow/docs_src/install/install_sources.md @@ -350,10 +350,10 @@ Invoke `pip install` to install that pip package. The filename of the `.whl` file depends on your platform. For example, the following command will install the pip package -for TensorFlow 1.6.0 on Linux: +for TensorFlow 1.7.0rc1 on Linux: <pre> -$ <b>sudo pip install /tmp/tensorflow_pkg/tensorflow-1.6.0-py2-none-any.whl</b> +$ <b>sudo pip install /tmp/tensorflow_pkg/tensorflow-1.7.0rc1-py2-none-any.whl</b> </pre> ## Validate your installation @@ -450,6 +450,8 @@ Stack Overflow and specify the `tensorflow` tag. **Linux** <table> <tr><th>Version:</th><th>CPU/GPU:</th><th>Python Version:</th><th>Compiler:</th><th>Build Tools:</th><th>cuDNN:</th><th>CUDA:</th></tr> +<tr><td>tensorflow-1.7.0rc1</td><td>CPU</td><td>2.7, 3.3-3.6</td><td>GCC 4.8</td><td>Bazel 0.10.0</td><td>N/A</td><td>N/A</td></tr> +<tr><td>tensorflow_gpu-1.7.0rc1</td><td>GPU</td><td>2.7, 3.3-3.6</td><td>GCC 4.8</td><td>Bazel 0.9.0</td><td>7</td><td>9</td></tr> <tr><td>tensorflow-1.6.0</td><td>CPU</td><td>2.7, 3.3-3.6</td><td>GCC 4.8</td><td>Bazel 0.9.0</td><td>N/A</td><td>N/A</td></tr> <tr><td>tensorflow_gpu-1.6.0</td><td>GPU</td><td>2.7, 3.3-3.6</td><td>GCC 4.8</td><td>Bazel 0.9.0</td><td>7</td><td>9</td></tr> <tr><td>tensorflow-1.5.0</td><td>CPU</td><td>2.7, 3.3-3.6</td><td>GCC 4.8</td><td>Bazel 0.8.0</td><td>N/A</td><td>N/A</td></tr> @@ -469,6 +471,7 @@ Stack Overflow and specify the `tensorflow` tag. **Mac** <table> <tr><th>Version:</th><th>CPU/GPU:</th><th>Python Version:</th><th>Compiler:</th><th>Build Tools:</th><th>cuDNN:</th><th>CUDA:</th></tr> +<tr><td>tensorflow-1.7.0rc1</td><td>CPU</td><td>2.7, 3.3-3.6</td><td>Clang from xcode</td><td>Bazel 0.10.1</td><td>N/A</td><td>N/A</td></tr> <tr><td>tensorflow-1.6.0</td><td>CPU</td><td>2.7, 3.3-3.6</td><td>Clang from xcode</td><td>Bazel 0.8.1</td><td>N/A</td><td>N/A</td></tr> <tr><td>tensorflow-1.5.0</td><td>CPU</td><td>2.7, 3.3-3.6</td><td>Clang from xcode</td><td>Bazel 0.8.1</td><td>N/A</td><td>N/A</td></tr> <tr><td>tensorflow-1.4.0</td><td>CPU</td><td>2.7, 3.3-3.6</td><td>Clang from xcode</td><td>Bazel 0.5.4</td><td>N/A</td><td>N/A</td></tr> @@ -483,6 +486,8 @@ Stack Overflow and specify the `tensorflow` tag. **Windows** <table> <tr><th>Version:</th><th>CPU/GPU:</th><th>Python Version:</th><th>Compiler:</th><th>Build Tools:</th><th>cuDNN:</th><th>CUDA:</th></tr> +<tr><td>tensorflow-1.7.0rc1</td><td>CPU</td><td>3.5-3.6</td><td>MSVC 2015 update 3</td><td>Cmake v3.6.3</td><td>N/A</td><td>N/A</td></tr> +<tr><td>tensorflow_gpu-1.7.0rc1</td><td>GPU</td><td>3.5-3.6</td><td>MSVC 2015 update 3</td><td>Cmake v3.6.3</td><td>7</td><td>9</td></tr> <tr><td>tensorflow-1.6.0</td><td>CPU</td><td>3.5-3.6</td><td>MSVC 2015 update 3</td><td>Cmake v3.6.3</td><td>N/A</td><td>N/A</td></tr> <tr><td>tensorflow_gpu-1.6.0</td><td>GPU</td><td>3.5-3.6</td><td>MSVC 2015 update 3</td><td>Cmake v3.6.3</td><td>7</td><td>9</td></tr> <tr><td>tensorflow-1.5.0</td><td>CPU</td><td>3.5-3.6</td><td>MSVC 2015 update 3</td><td>Cmake v3.6.3</td><td>N/A</td><td>N/A</td></tr> diff --git a/tensorflow/docs_src/mobile/optimizing.md b/tensorflow/docs_src/mobile/optimizing.md index ca9cb043e9..778e4d3a62 100644 --- a/tensorflow/docs_src/mobile/optimizing.md +++ b/tensorflow/docs_src/mobile/optimizing.md @@ -233,6 +233,8 @@ order by how long they took. From left to right, the columns are: - The cumulative total time of this and the previous ops in the table. This is handy for understanding what the distribution of work is across the layers, to see if just a few of the nodes are taking up most of the time. + +- The amount of memory consumed by outputs of this type of op. - Name of the node. diff --git a/tensorflow/docs_src/mobile/prepare_models.md b/tensorflow/docs_src/mobile/prepare_models.md index 360ee302aa..8b22c04d87 100644 --- a/tensorflow/docs_src/mobile/prepare_models.md +++ b/tensorflow/docs_src/mobile/prepare_models.md @@ -60,7 +60,7 @@ and serialized as protocol buffers: the `NodeDef`, so if all the `Variable` weights are converted to `Const` nodes, then we only need a single `GraphDef` file to hold the model architecture and the weights. Freezing the graph handles the process of loading the - checkpoints, and then converts all Consts to Variables. You can then load the + checkpoints, and then converts all Variables to Consts. You can then load the resulting file in a single call, without having to restore variable values from checkpoints. One thing to watch out for with `GraphDef` files is that sometimes they’re stored in text format for easy inspection. These versions diff --git a/tensorflow/python/BUILD b/tensorflow/python/BUILD index 4f61c01f65..a0dd409205 100644 --- a/tensorflow/python/BUILD +++ b/tensorflow/python/BUILD @@ -1065,7 +1065,7 @@ py_test( py_test( name = "framework_importer_test", - size = "medium", + size = "large", srcs = ["framework/importer_test.py"], main = "framework/importer_test.py", srcs_version = "PY2AND3", diff --git a/tensorflow/python/kernel_tests/array_ops_test.py b/tensorflow/python/kernel_tests/array_ops_test.py index d0ba8020c1..64c1760d5e 100644 --- a/tensorflow/python/kernel_tests/array_ops_test.py +++ b/tensorflow/python/kernel_tests/array_ops_test.py @@ -315,21 +315,39 @@ class ReverseV2Test(test_util.TensorFlowTestCase): self.assertAllEqual(x_tf_4, np.asarray(x_np)[:, ::-1]) self.assertAllEqual(x_tf_5, np.asarray(x_np)[::-1, ::-1]) + # This test covers the axis validation in the shape function + # (no eval()) + def testInvalidAxis(self): + x_np = np.array([[1, 2, 3], [4, 5, 6]], dtype=np.float32) + with self.assertRaisesRegexp(ValueError, + "is out of valid range"): + array_ops.reverse_v2(x_np, [-30]) + with self.assertRaisesRegexp(ValueError, + "is out of valid range"): + array_ops.reverse_v2(x_np, [2]) + with self.assertRaisesRegexp(ValueError, + "axis 0 specified more than once"): + array_ops.reverse_v2(x_np, [0, -2]) + # This is the version of reverse that uses axis indices rather than # bool tensors # TODO(b/32254538): Change this test to use array_ops.reverse + # + # Note: this test passes placeholder as constant axis is validated + # in shape function (see testInvalidAxis) def testInvalid(self): x_np = np.array([[1, 2, 3], [4, 5, 6]], dtype=np.float32) + axis = array_ops.placeholder(dtypes.int32) with self.test_session(): with self.assertRaisesRegexp(errors_impl.InvalidArgumentError, "is out of valid range"): - array_ops.reverse_v2(x_np, [-30]).eval() + array_ops.reverse_v2(x_np, axis).eval(feed_dict={axis: [-30]}) with self.assertRaisesRegexp(errors_impl.InvalidArgumentError, "is out of valid range"): - array_ops.reverse_v2(x_np, [2]).eval() + array_ops.reverse_v2(x_np, axis).eval(feed_dict={axis: [2]}) with self.assertRaisesRegexp(errors_impl.InvalidArgumentError, "axis 0 specified more than once"): - array_ops.reverse_v2(x_np, [0, -2]).eval() + array_ops.reverse_v2(x_np, axis).eval(feed_dict={axis: [0, -2]}) def testReverse1DimAuto(self): for dtype in [ @@ -890,7 +908,7 @@ class StridedSliceAssignChecker(object): var = resource_variable_ops.ResourceVariable(self.x) else: var = variables.Variable(self.x) - sess.run(variables.initialize_variables([var])) + sess.run(variables.variables_initializer([var])) val = sess.run(var[index].assign(value)) # val_copy is used to check that tf.assign works equivalently to the # assign method above. diff --git a/tensorflow/python/kernel_tests/testdata/BUILD b/tensorflow/python/kernel_tests/testdata/BUILD index a4a0dfc139..45264c773a 100644 --- a/tensorflow/python/kernel_tests/testdata/BUILD +++ b/tensorflow/python/kernel_tests/testdata/BUILD @@ -1,7 +1,7 @@ # Data files for kernel tests. package( - default_visibility = ["//tensorflow:internal"], + default_visibility = ["//visibility:public"], ) licenses(["notice"]) # Apache 2.0 diff --git a/tensorflow/python/kernel_tests/xent_op_test.py b/tensorflow/python/kernel_tests/xent_op_test.py index e3e120a4eb..60c726d54c 100644 --- a/tensorflow/python/kernel_tests/xent_op_test.py +++ b/tensorflow/python/kernel_tests/xent_op_test.py @@ -18,10 +18,16 @@ from __future__ import absolute_import from __future__ import division from __future__ import print_function +import itertools +import sys + import numpy as np +from tensorflow.python.client import session from tensorflow.python.framework import constant_op from tensorflow.python.framework import dtypes +from tensorflow.python.framework import ops +from tensorflow.python.ops import array_ops from tensorflow.python.ops import gen_nn_ops from tensorflow.python.ops import gradient_checker from tensorflow.python.ops import gradients_impl @@ -88,7 +94,7 @@ class XentTest(test.TestCase): 4.]]]).astype(dtype) np_labels = np.array([[[0., 0., 0., 1.]], [[0., .5, .5, 0.]]]).astype(dtype) - self.assertRaisesRegexp(ValueError, "must be rank 2", + self.assertRaisesRegexp(ValueError, "rank 2, but is rank 3", gen_nn_ops.softmax_cross_entropy_with_logits, np_features, np_labels) @@ -128,6 +134,24 @@ class XentTest(test.TestCase): self.assertAllClose( np.array([1.3862, 1.9401]), np_loss, rtol=1.e-3, atol=1.e-3) + def testShapeBroadcast(self): + np_f = np.array([[1., 2., 3., 4.], + [1., 2., 3., 4.]]).astype(np.float32) + np_l = np.array([[0., 0., 0., 1.], + [0., .5, .5, 0.]]).astype(np.float32) + np_loss, np_backprop = self._npXent(np_f, np_l) + tf_f = constant_op.constant( + np.array([[1., 2., 3., 4.]]).astype(np.float32)) + tf_l = constant_op.constant( + np.array([[0., 0., 0., 1.], [0., .5, .5, 0.]]).astype(np.float32)) + for use_gpu in [False, True]: + with self.test_session(use_gpu=use_gpu) as sess: + loss, backprop = gen_nn_ops.softmax_cross_entropy_with_logits( + tf_f, tf_l) + tf_loss, tf_backprop = sess.run([loss, backprop]) + self.assertAllCloseAccordingToType(np_loss, tf_loss) + self.assertAllCloseAccordingToType(np_backprop, tf_backprop) + def testShapeMismatch(self): with self.test_session(): with self.assertRaises(ValueError): @@ -260,5 +284,60 @@ class XentTest(test.TestCase): self.assertAllEqual(np_loss, tf_loss) +class XentBenchmark(test.Benchmark): + + def benchmarkZeroDimension(self): + for (m, n, p, use_gpu) in itertools.product( + [128], + [10, 100, 1000, 10000, 100000], + [0.001, 0.01, 0.5, 0.99, 1.0], + [False]): + k = int(p * n) + if k == 0: + continue + name = "zero_dimension_m_%d_n_%d_k_%g_use_gpu_%s" % (m, n, k, use_gpu) + device = "/%s:0" % ("gpu" if use_gpu else "cpu") + with ops.Graph().as_default(): + with ops.device(device): + labels = array_ops.zeros([0, 2, 4], dtype=dtypes.float32) + logits = array_ops.zeros([0, 2, 4], dtype=dtypes.float32) + op = nn_ops.softmax_cross_entropy_with_logits( + labels=labels, logits=logits) + with session.Session() as sess: + r = self.run_op_benchmark(sess, op, min_iters=100, name=name) + gb_processed_input = m * n / 1.0e9 + throughput = gb_processed_input / r["wall_time"] + print("Benchmark: %s \t wall_time: %0.03g s \t " + "Throughput: %0.03g GB/s" % (name, r["wall_time"], throughput)) + sys.stdout.flush() + + def benchmarkSingleClass(self): + for (m, n, p, use_gpu) in itertools.product( + [128], + [10, 100, 1000, 10000, 100000], + [0.001, 0.01, 0.5, 0.99, 1.0], + [False]): + k = int(p * n) + if k == 0: + continue + name = "single_class_m_%d_n_%d_k_%g_use_gpu_%s" % (m, n, k, use_gpu) + device = "/%s:0" % ("gpu" if use_gpu else "cpu") + with ops.Graph().as_default(): + with ops.device(device): + labels = constant_op.constant([[1.], [-1.], [0.]], + dtype=dtypes.float32) + logits = constant_op.constant([[-1.], [0.], [1.]], + dtype=dtypes.float32) + op = nn_ops.softmax_cross_entropy_with_logits( + labels=labels, logits=logits) + with session.Session() as sess: + r = self.run_op_benchmark(sess, op, min_iters=100, name=name) + gb_processed_input = m * n / 1.0e9 + throughput = gb_processed_input / r["wall_time"] + print("Benchmark: %s \t wall_time: %0.03g s \t " + "Throughput: %0.03g GB/s" % (name, r["wall_time"], throughput)) + sys.stdout.flush() + + if __name__ == "__main__": test.main() diff --git a/tensorflow/python/layers/convolutional.py b/tensorflow/python/layers/convolutional.py index 74e7c63fb3..2d99b1688f 100644 --- a/tensorflow/python/layers/convolutional.py +++ b/tensorflow/python/layers/convolutional.py @@ -180,6 +180,8 @@ class _Conv(base.Layer): # bias_add when computing gradients. To use bias_add, we collapse Z # and Y into a single dimension to obtain a 4D input tensor. outputs_shape = outputs.shape.as_list() + if outputs_shape[0] is None: + outputs_shape[0] = -1 outputs_4d = array_ops.reshape(outputs, [outputs_shape[0], outputs_shape[1], outputs_shape[2] * outputs_shape[3], diff --git a/tensorflow/python/layers/convolutional_test.py b/tensorflow/python/layers/convolutional_test.py index 160e732b67..cdb42f5bd1 100644 --- a/tensorflow/python/layers/convolutional_test.py +++ b/tensorflow/python/layers/convolutional_test.py @@ -325,6 +325,12 @@ class ConvTest(test.TestCase): self.assertEqual(conv3d.kernel_constraint, k_constraint) self.assertEqual(conv3d.bias_constraint, b_constraint) + def testConv3DChannelsFirst(self): + # Test case for GitHub issue 15655 + images = array_ops.placeholder( + dtype=dtypes.float32, shape=[None, 1, 32, 32, 32]) + conv_layers.conv3d(images, 32, 9, data_format='channels_first') + @test_util.with_c_api class SeparableConv1DTest(test.TestCase): diff --git a/tensorflow/python/ops/linalg_ops.py b/tensorflow/python/ops/linalg_ops.py index 5b4fb4f7c8..170861b43f 100644 --- a/tensorflow/python/ops/linalg_ops.py +++ b/tensorflow/python/ops/linalg_ops.py @@ -429,7 +429,7 @@ def svd(tensor, full_matrices=False, compute_uv=True, name=None): u, s, v_adj = np.linalg.svd(a, full_matrices=False) np_a_approx = np.dot(u, np.dot(np.diag(s), v_adj)) # tf_a_approx and np_a_approx should be numerically close. - ```` + ``` @end_compatibility """ s, u, v = gen_linalg_ops.svd( diff --git a/tensorflow/python/training/monitored_session.py b/tensorflow/python/training/monitored_session.py index 6c5c9e01a7..4ce6f6d002 100644 --- a/tensorflow/python/training/monitored_session.py +++ b/tensorflow/python/training/monitored_session.py @@ -281,13 +281,14 @@ def MonitoredTrainingSession(master='', # pylint: disable=invalid-name scaffold=None, hooks=None, chief_only_hooks=None, - save_checkpoint_secs=600, + save_checkpoint_secs=USE_DEFAULT, save_summaries_steps=USE_DEFAULT, save_summaries_secs=USE_DEFAULT, config=None, stop_grace_period_secs=120, log_step_count_steps=100, - max_wait_secs=7200): + max_wait_secs=7200, + save_checkpoint_steps=USE_DEFAULT): """Creates a `MonitoredSession` for training. For a chief, this utility sets proper session initializer/restorer. It also @@ -310,8 +311,10 @@ def MonitoredTrainingSession(master='', # pylint: disable=invalid-name chief_only_hooks: list of `SessionRunHook` objects. Activate these hooks if `is_chief==True`, ignore otherwise. save_checkpoint_secs: The frequency, in seconds, that a checkpoint is saved - using a default checkpoint saver. If `save_checkpoint_secs` is set to - `None`, then the default checkpoint saver isn't used. + using a default checkpoint saver. If both `save_checkpoint_steps` and + `save_checkpoint_secs` are set to `None`, then the default checkpoint + saver isn't used. If both are provided, then only `save_checkpoint_secs` + is used. Default 600. save_summaries_steps: The frequency, in number of global steps, that the summaries are written to disk using a default summary saver. If both `save_summaries_steps` and `save_summaries_secs` are set to `None`, then @@ -330,6 +333,11 @@ def MonitoredTrainingSession(master='', # pylint: disable=invalid-name become available. This should be kept relatively short to help detect incorrect code, but sometimes may need to be increased if the chief takes a while to start up. + save_checkpoint_steps: The frequency, in number of global steps, that a + checkpoint is saved using a default checkpoint saver. If both + `save_checkpoint_steps` and `save_checkpoint_secs` are set to `None`, then + the default checkpoint saver isn't used. If both are provided, then only + `save_checkpoint_secs` is used. Default not enabled. Returns: A `MonitoredSession` object. @@ -342,6 +350,15 @@ def MonitoredTrainingSession(master='', # pylint: disable=invalid-name elif save_summaries_steps == USE_DEFAULT: save_summaries_steps = None + if (save_checkpoint_steps == USE_DEFAULT and + save_checkpoint_secs == USE_DEFAULT): + save_checkpoint_steps = None + save_checkpoint_secs = 600 + elif save_checkpoint_secs == USE_DEFAULT: + save_checkpoint_secs = None + elif save_checkpoint_steps == USE_DEFAULT: + save_checkpoint_steps = None + scaffold = scaffold or Scaffold() if not is_chief: session_creator = WorkerSessionCreator( @@ -374,9 +391,13 @@ def MonitoredTrainingSession(master='', # pylint: disable=invalid-name save_steps=save_summaries_steps, save_secs=save_summaries_secs, output_dir=checkpoint_dir)) - if save_checkpoint_secs and save_checkpoint_secs > 0: + if (save_checkpoint_secs and save_checkpoint_secs > 0) or ( + save_checkpoint_steps and save_checkpoint_steps > 0): all_hooks.append(basic_session_run_hooks.CheckpointSaverHook( - checkpoint_dir, save_secs=save_checkpoint_secs, scaffold=scaffold)) + checkpoint_dir, + save_steps=save_checkpoint_steps, + save_secs=save_checkpoint_secs, + scaffold=scaffold)) if hooks: all_hooks.extend(hooks) diff --git a/tensorflow/python/training/monitored_session_test.py b/tensorflow/python/training/monitored_session_test.py index 159b2d5c16..3806056f01 100644 --- a/tensorflow/python/training/monitored_session_test.py +++ b/tensorflow/python/training/monitored_session_test.py @@ -282,6 +282,42 @@ class MonitoredTrainingSessionTest(test.TestCase): is_chief=True, checkpoint_dir=logdir) as session: self.assertEqual(2, session.run(gstep)) + def test_save_checkpoint_steps(self): + logdir = _test_dir(self.get_temp_dir(), 'test_save_checkpoint_steps') + with ops.Graph().as_default(): + gstep = variables_lib.get_or_create_global_step() + new_gstep = state_ops.assign_add(gstep, 1) + with monitored_session.MonitoredTrainingSession( + is_chief=True, + checkpoint_dir=logdir, + save_checkpoint_steps=100, + log_step_count_steps=10) as session: + for _ in range(100): + session.run(new_gstep) + # A restart will find the checkpoint and recover automatically. + with monitored_session.MonitoredTrainingSession( + is_chief=True, checkpoint_dir=logdir) as session: + self.assertEqual(100, session.run(gstep)) + + def test_save_checkpoint_secs(self): + logdir = _test_dir(self.get_temp_dir(), 'test_save_checkpoint_secs') + with ops.Graph().as_default(): + gstep = variables_lib.get_or_create_global_step() + new_gstep = state_ops.assign_add(gstep, 1) + with monitored_session.MonitoredTrainingSession( + is_chief=True, + checkpoint_dir=logdir, + save_checkpoint_secs=0.1, + log_step_count_steps=10) as session: + session.run(new_gstep) + time.sleep(0.2) + for _ in range(10): + session.run(new_gstep) + # A restart will find the checkpoint and recover automatically. + with monitored_session.MonitoredTrainingSession( + is_chief=True, checkpoint_dir=logdir) as session: + self.assertEqual(11, session.run(gstep)) + def test_summaries_steps(self): logdir = _test_dir(self.get_temp_dir(), 'test_summaries_steps') with ops.Graph().as_default(): diff --git a/tensorflow/tensorflow.bzl b/tensorflow/tensorflow.bzl index 2d3cb415fe..fcc57d506e 100644 --- a/tensorflow/tensorflow.bzl +++ b/tensorflow/tensorflow.bzl @@ -22,6 +22,7 @@ load( load( "//third_party/mkl:build_defs.bzl", "if_mkl", + "if_mkl_lnx_x64" ) def register_extension_info(**kwargs): @@ -202,7 +203,8 @@ def tf_copts(android_optimization_level_override="-O2", is_external=False): "-ftemplate-depth=900"]) + if_cuda(["-DGOOGLE_CUDA=1"]) + if_tensorrt(["-DGOOGLE_TENSORRT=1"]) - + if_mkl(["-DINTEL_MKL=1", "-DEIGEN_USE_VML", "-fopenmp",]) + + if_mkl(["-DINTEL_MKL=1", "-DEIGEN_USE_VML"]) + + if_mkl_lnx_x64(["-fopenmp"]) + if_android_arm(["-mfpu=neon"]) + if_linux_x86_64(["-msse3"]) + if_ios_x86_64(["-msse4.1"]) diff --git a/tensorflow/tools/api/golden/tensorflow.train.pbtxt b/tensorflow/tools/api/golden/tensorflow.train.pbtxt index c75ee474aa..bec72e1e60 100644 --- a/tensorflow/tools/api/golden/tensorflow.train.pbtxt +++ b/tensorflow/tools/api/golden/tensorflow.train.pbtxt @@ -238,7 +238,7 @@ tf_module { } member_method { name: "MonitoredTrainingSession" - argspec: "args=[\'master\', \'is_chief\', \'checkpoint_dir\', \'scaffold\', \'hooks\', \'chief_only_hooks\', \'save_checkpoint_secs\', \'save_summaries_steps\', \'save_summaries_secs\', \'config\', \'stop_grace_period_secs\', \'log_step_count_steps\', \'max_wait_secs\'], varargs=None, keywords=None, defaults=[\'\', \'True\', \'None\', \'None\', \'None\', \'None\', \'600\', \'<object object instance>\', \'<object object instance>\', \'None\', \'120\', \'100\', \'7200\'], " + argspec: "args=[\'master\', \'is_chief\', \'checkpoint_dir\', \'scaffold\', \'hooks\', \'chief_only_hooks\', \'save_checkpoint_secs\', \'save_summaries_steps\', \'save_summaries_secs\', \'config\', \'stop_grace_period_secs\', \'log_step_count_steps\', \'max_wait_secs\', \'save_checkpoint_steps\'], varargs=None, keywords=None, defaults=[\'\', \'True\', \'None\', \'None\', \'None\', \'None\', \'<object object instance>\', \'<object object instance>\', \'<object object instance>\', \'None\', \'120\', \'100\', \'7200\', \'<object object instance>\'], " } member_method { name: "NewCheckpointReader" diff --git a/tensorflow/tools/ci_build/osx/libtensorflow_cpu.sh b/tensorflow/tools/ci_build/osx/libtensorflow_cpu.sh index e1b56b9a25..7d471b4703 100755 --- a/tensorflow/tools/ci_build/osx/libtensorflow_cpu.sh +++ b/tensorflow/tools/ci_build/osx/libtensorflow_cpu.sh @@ -31,5 +31,5 @@ export TF_NEED_OPENCL_SYCL=0 export TF_NEED_MKL=0 export COMPUTECPP_PATH="/usr/local" -export PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin" +export PATH="$PATH:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin" build_libtensorflow_tarball "-cpu-darwin-$(uname -m)" diff --git a/tensorflow/tools/docker/Dockerfile.devel b/tensorflow/tools/docker/Dockerfile.devel index 22c73c3fe1..11f476d12c 100644 --- a/tensorflow/tools/docker/Dockerfile.devel +++ b/tensorflow/tools/docker/Dockerfile.devel @@ -70,7 +70,7 @@ RUN mkdir /bazel && \ # Download and build TensorFlow. WORKDIR /tensorflow -RUN git clone --branch=r1.6 --depth=1 https://github.com/tensorflow/tensorflow.git . +RUN git clone --branch=r1.7 --depth=1 https://github.com/tensorflow/tensorflow.git . # TODO(craigcitro): Don't install the pip package, since it makes it # more difficult to experiment with local changes. Instead, just add diff --git a/tensorflow/tools/docker/Dockerfile.devel-cpu-mkl b/tensorflow/tools/docker/Dockerfile.devel-cpu-mkl index 3690e7dfe5..037d13116e 100644 --- a/tensorflow/tools/docker/Dockerfile.devel-cpu-mkl +++ b/tensorflow/tools/docker/Dockerfile.devel-cpu-mkl @@ -3,7 +3,7 @@ FROM tensorflow/tensorflow:latest-devel LABEL maintainer="Clayne Robison<clayne.b.robison@intel.com>" # These arguments are parameterized. Use --build-args to override. -ARG TF_BRANCH=r1.6 +ARG TF_BRANCH=r1.7 ARG WHL_DIR=/whl RUN apt-get update && apt-get install -y --no-install-recommends \ diff --git a/tensorflow/tools/docker/Dockerfile.devel-gpu b/tensorflow/tools/docker/Dockerfile.devel-gpu index 69ba340f92..1fcb6428b2 100644 --- a/tensorflow/tools/docker/Dockerfile.devel-gpu +++ b/tensorflow/tools/docker/Dockerfile.devel-gpu @@ -79,7 +79,7 @@ RUN mkdir /bazel && \ # Download and build TensorFlow. WORKDIR /tensorflow -RUN git clone --branch=r1.6 --depth=1 https://github.com/tensorflow/tensorflow.git . +RUN git clone --branch=r1.7 --depth=1 https://github.com/tensorflow/tensorflow.git . # Configure the build for our CUDA configuration. ENV CI_BUILD_PYTHON python diff --git a/tensorflow/tools/lib_package/BUILD b/tensorflow/tools/lib_package/BUILD index 3fbdb5cacd..0ede8c6370 100644 --- a/tensorflow/tools/lib_package/BUILD +++ b/tensorflow/tools/lib_package/BUILD @@ -138,7 +138,6 @@ genrule( "@zlib_archive//:zlib.h", ] + if_mkl([ "//third_party/mkl:LICENSE", - "@mkl//:LICENSE", ]), outs = ["include/tensorflow/c/LICENSE"], cmd = "$(location :concat_licenses.sh) $(SRCS) >$@", @@ -176,7 +175,6 @@ genrule( "@zlib_archive//:zlib.h", ] + if_mkl([ "//third_party/mkl:LICENSE", - "@mkl//:LICENSE", ]), outs = ["include/tensorflow/jni/LICENSE"], cmd = "$(location :concat_licenses.sh) $(SRCS) >$@", diff --git a/tensorflow/tools/pip_package/BUILD b/tensorflow/tools/pip_package/BUILD index dd75eda231..62fec2c402 100644 --- a/tensorflow/tools/pip_package/BUILD +++ b/tensorflow/tools/pip_package/BUILD @@ -127,7 +127,6 @@ filegroup( "@org_python_pypi_backports_weakref//:LICENSE", ] + if_mkl([ "//third_party/mkl:LICENSE", - "@mkl//:LICENSE", ]) + if_not_windows([ "@nccl_archive//:LICENSE.txt", ]) + tf_additional_license_deps(), diff --git a/tensorflow/tools/pip_package/setup.py b/tensorflow/tools/pip_package/setup.py index e0152da4df..365e8d6b08 100644 --- a/tensorflow/tools/pip_package/setup.py +++ b/tensorflow/tools/pip_package/setup.py @@ -29,7 +29,7 @@ from setuptools.dist import Distribution # This version string is semver compatible, but incompatible with pip. # For pip, we will remove all '-' characters from this string, and use the # result for pip. -_VERSION = '1.6.0' +_VERSION = '1.7.0-rc1' REQUIRED_PACKAGES = [ 'absl-py >= 0.1.6', @@ -39,7 +39,7 @@ REQUIRED_PACKAGES = [ 'numpy >= 1.13.3', 'six >= 1.10.0', 'protobuf >= 3.4.0', - 'tensorboard >= 1.6.0, < 1.7.0', + 'tensorboard >= 1.7.0, < 1.8.0', 'termcolor >= 1.1.0', ] @@ -62,7 +62,7 @@ else: if 'tf_nightly' in project_name: for i, pkg in enumerate(REQUIRED_PACKAGES): if 'tensorboard' in pkg: - REQUIRED_PACKAGES[i] = 'tb-nightly >= 1.7.0a0, < 1.8.0a0' + REQUIRED_PACKAGES[i] = 'tb-nightly >= 1.8.0a0, < 1.9.0a0' break # weakref.finalize and enum were introduced in Python 3.4 diff --git a/tensorflow/workspace.bzl b/tensorflow/workspace.bzl index 9fcbfb664b..5f6e717532 100644 --- a/tensorflow/workspace.bzl +++ b/tensorflow/workspace.bzl @@ -15,6 +15,11 @@ load("@io_bazel_rules_closure//closure/private:java_import_external.bzl", "java_ load("@io_bazel_rules_closure//closure:defs.bzl", "filegroup_external") +# Sanitize a dependency so that it works correctly from code that includes +# TensorFlow as a submodule. +def clean_dep(dep): + return str(Label(dep)) + # If TensorFlow is linked as a submodule. # path_prefix is no longer used. # tf_repo_name is thought to be under consideration. @@ -32,17 +37,37 @@ def tf_workspace(path_prefix="", tf_repo_name=""): arm_compiler_configure( name="local_config_arm_compiler", remote_config_repo="../arm_compiler", - build_file = str(Label("//third_party/toolchains/cpus/arm:BUILD"))) + build_file = clean_dep("//third_party/toolchains/cpus/arm:BUILD")) mkl_repository( - name = "mkl", + name = "mkl_linux", + urls = [ + "https://mirror.bazel.build/intel/mkl-dnn/releases/download/v0.12/mklml_lnx_2018.0.1.20171227.tgz", + "https://github.com/intel/mkl-dnn/releases/download/v0.12/mklml_lnx_2018.0.1.20171227.tgz", + ], + sha256 = "feacc3d82565c1231470359b42c696236fae873704e0b013436afba5fd4fd30f", + strip_prefix = "mklml_lnx_2018.0.1.20171227", + build_file = clean_dep("//third_party/mkl:mkl.BUILD") + ) + mkl_repository( + name = "mkl_windows", + urls = [ + "https://mirror.bazel.build/intel/mkl-dnn/releases/download/v0.12/mklml_win_2018.0.1.20171227.zip", + "https://github.com/intel/mkl-dnn/releases/download/v0.12/mklml_win_2018.0.1.20171227.zip" + ], + sha256 = "24bae8d7b22b431a654acadea43f2243c46ae6b1e5a73a4a936825f31d284ee4", + strip_prefix = "mklml_win_2018.0.1.20171227", + build_file = clean_dep("//third_party/mkl:mkl.BUILD") + ) + mkl_repository( + name = "mkl_darwin", urls = [ - "https://mirror.bazel.build/github.com/01org/mkl-dnn/releases/download/v0.11/mklml_lnx_2018.0.1.20171007.tgz", - "https://github.com/01org/mkl-dnn/releases/download/v0.11/mklml_lnx_2018.0.1.20171007.tgz", + "https://mirror.bazel.build/intel/mkl-dnn/releases/download/v0.12/mklml_mac_2018.0.1.20171227.tgz", + "https://github.com/intel/mkl-dnn/releases/download/v0.12/mklml_mac_2018.0.1.20171227.tgz" ], - sha256 = "6b07cb7e5451db67c2e31e785ae458b18f7f363c60a61685488f69e9ae7199d4", - strip_prefix = "mklml_lnx_2018.0.1.20171007", - build_file = str(Label("//third_party/mkl:mkl.BUILD")), + sha256 = "0e954ec6fd3dc5e37f64c4043f6b5613dd687558da3df1028b3b7c29ff5cf77f", + strip_prefix = "mklml_mac_2018.0.1.20171227", + build_file = clean_dep("//third_party/mkl:mkl.BUILD") ) if path_prefix: @@ -52,12 +77,12 @@ def tf_workspace(path_prefix="", tf_repo_name=""): tf_http_archive( name = "mkl_dnn", urls = [ - "https://mirror.bazel.build/github.com/01org/mkl-dnn/archive/e0bfcaa7fcb2b1e1558f5f0676933c1db807a729.tar.gz", - "https://github.com/01org/mkl-dnn/archive/e0bfcaa7fcb2b1e1558f5f0676933c1db807a729.tar.gz", + "https://mirror.bazel.build/github.com/intel/mkl-dnn/archive/v0.12.tar.gz", + "https://github.com/intel/mkl-dnn/archive/v0.12.tar.gz", ], - sha256 = "02e244f63dd95402691a361392504c143eede9a89043426f174836638a9cbf09", - strip_prefix = "mkl-dnn-e0bfcaa7fcb2b1e1558f5f0676933c1db807a729", - build_file = str(Label("//third_party/mkl_dnn:mkldnn.BUILD")), + sha256 = "86fa2a8c12a56e3b725945acedeaa82492746be02545aba6d710f097e013e19e", + strip_prefix = "mkl-dnn-0.12", + build_file = clean_dep("//third_party/mkl_dnn:mkldnn.BUILD"), ) tf_http_archive( @@ -68,7 +93,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "5996380e3e8b981f55d1c8d58e709c00dbb4806ba367be75d0925a68cc2f6478", strip_prefix = "abseil-cpp-720c017e30339fd1786ce4aac68bc8559736e53f", - build_file = str(Label("//third_party:com_google_absl.BUILD")), + build_file = clean_dep("//third_party:com_google_absl.BUILD"), ) tf_http_archive( @@ -79,8 +104,8 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "0cadb31a35b514bf2dfd6b5d38205da94ef326ec6908fc3fd7c269948467214f", strip_prefix = "eigen-eigen-2355b229ea4c", - build_file = str(Label("//third_party:eigen.BUILD")), - patch_file = str(Label("//third_party:eigen_fix_cuda_compilation.patch")) + build_file = clean_dep("//third_party:eigen.BUILD"), + patch_file = clean_dep("//third_party:eigen_fix_cuda_compilation.patch") ) tf_http_archive( @@ -93,7 +118,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): # remove the whitelist entry in third_party/repo.bzl. # "https://github.com/raspberrypi/tools/archive/0e906ebc527eab1cdbf7adabff5b474da9562e9f.tar.gz", ], - build_file = str(Label("//:arm_compiler.BUILD")), + build_file = clean_dep("//:arm_compiler.BUILD"), ) tf_http_archive( @@ -104,7 +129,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "2ade869c3f42f23b5263c7d594aa3c7e5e61ac6a3afcaf5d6e42899d2a7986ce", strip_prefix = "libxsmm-1.8.1", - build_file = str(Label("//third_party:libxsmm.BUILD")), + build_file = clean_dep("//third_party:libxsmm.BUILD"), ) tf_http_archive( @@ -117,7 +142,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "932075525642b04ac6f1b50589f1df5cd72ec2f448b721fd32234cf183f0e755", strip_prefix = "or-tools-253f7955c6a1fd805408fba2e42ac6d45b312d15/src", - build_file = str(Label("//third_party:ortools.BUILD")), + build_file = clean_dep("//third_party:ortools.BUILD"), ) tf_http_archive( @@ -149,7 +174,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "6560547c63e4af82b0f202cb710ceabb3f21347a4b996db565a411da5b17aba0", strip_prefix = "farmhash-816a4ae622e964763ca0862d9dbd19324a1eaf45", - build_file = str(Label("//third_party:farmhash.BUILD")), + build_file = clean_dep("//third_party:farmhash.BUILD"), ) tf_http_archive( @@ -160,7 +185,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "0f30a15b1566d93f146c8d149878a06e91d9bb7ec2cfd76906df62a82be4aac9", strip_prefix = "highwayhash-dfcb97ca4fe9277bf9dc1802dd979b071896453b", - build_file = str(Label("//third_party:highwayhash.BUILD")), + build_file = clean_dep("//third_party:highwayhash.BUILD"), ) tf_http_archive( @@ -171,7 +196,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "00b0891c678c065446ca59bcee64719d0096d54d6886e6e472aeee2e170ae324", strip_prefix = "nasm-2.12.02", - build_file = str(Label("//third_party:nasm.BUILD")), + build_file = clean_dep("//third_party:nasm.BUILD"), ) tf_http_archive( @@ -182,7 +207,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "c15a9607892113946379ccea3ca8b85018301b200754f209453ab21674268e77", strip_prefix = "libjpeg-turbo-1.5.1", - build_file = str(Label("//third_party/jpeg:jpeg.BUILD")), + build_file = clean_dep("//third_party/jpeg:jpeg.BUILD"), ) tf_http_archive( @@ -193,7 +218,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "716c59c7dfc808a4c368f8ada526932be72b2fcea11dd85dc9d88b1df1dfe9c2", strip_prefix = "libpng-1.2.53", - build_file = str(Label("//third_party:png.BUILD")), + build_file = clean_dep("//third_party:png.BUILD"), ) tf_http_archive( @@ -204,7 +229,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "208780b3616f9de0aeb50822b7a8f5482f6515193859e91ed61637be6ad74fd4", strip_prefix = "sqlite-amalgamation-3200000", - build_file = str(Label("//third_party:sqlite.BUILD")), + build_file = clean_dep("//third_party:sqlite.BUILD"), ) tf_http_archive( @@ -215,7 +240,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "34a7377ba834397db019e8eb122e551a49c98f49df75ec3fcc92b9a794a4f6d1", strip_prefix = "giflib-5.1.4", - build_file = str(Label("//third_party:gif.BUILD")), + build_file = clean_dep("//third_party:gif.BUILD"), ) tf_http_archive( @@ -226,7 +251,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "105f8d68616f8248e24bf0e9372ef04d3cc10104f1980f54d57b2ce73a5ad56a", strip_prefix = "six-1.10.0", - build_file = str(Label("//third_party:six.BUILD")), + build_file = clean_dep("//third_party:six.BUILD"), ) tf_http_archive( @@ -237,7 +262,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "ff6d2e2962d834acb125cc4dcc80c54a8c17c253f4cc9d9c43b5102a560bb75d", strip_prefix = "astor-0.6.2", - build_file = str(Label("//third_party:astor.BUILD")), + build_file = clean_dep("//third_party:astor.BUILD"), ) tf_http_archive( @@ -248,7 +273,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "7068908321ecd2774f145193c4b34a11305bd104b4551b09273dfd1d6a374930", strip_prefix = "gast-0.2.0", - build_file = str(Label("//third_party:gast.BUILD")), + build_file = clean_dep("//third_party:gast.BUILD"), ) tf_http_archive( @@ -259,7 +284,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "1d6d69ce66211143803fbc56652b41d73b4a400a2891d7bf7a1cdf4c02de613b", strip_prefix = "termcolor-1.1.0", - build_file = str(Label("//third_party:termcolor.BUILD")), + build_file = clean_dep("//third_party:termcolor.BUILD"), ) tf_http_archive( @@ -280,7 +305,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "8813bf712a66b3d8b85dc289e1104ed220f1878cf981e2fe756dfaabe9a82892", strip_prefix = "backports.weakref-1.0rc1/src", - build_file = str(Label("//third_party:backports_weakref.BUILD")), + build_file = clean_dep("//third_party:backports_weakref.BUILD"), ) tf_http_archive( @@ -291,7 +316,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "2dadd04a2802de27e0fe5a19b76538f6da9d39ff244036afa00c1bba754de5ee", strip_prefix = "codegen-1.0", - build_file = str(Label("//third_party:codegen.BUILD")), + build_file = clean_dep("//third_party:codegen.BUILD"), ) filegroup_external( @@ -376,7 +401,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): "http://ftp.exim.org/pub/pcre/pcre-8.39.tar.gz", ], strip_prefix = "pcre-8.39", - build_file = str(Label("//third_party:pcre.BUILD")), + build_file = clean_dep("//third_party:pcre.BUILD"), ) tf_http_archive( @@ -388,7 +413,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): "http://pilotfiber.dl.sourceforge.net/project/swig/swig/swig-3.0.8/swig-3.0.8.tar.gz", ], strip_prefix = "swig-3.0.8", - build_file = str(Label("//third_party:swig.BUILD")), + build_file = clean_dep("//third_party:swig.BUILD"), ) tf_http_archive( @@ -399,7 +424,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): "https://curl.haxx.se/download/curl-7.49.1.tar.gz", ], strip_prefix = "curl-7.49.1", - build_file = str(Label("//third_party:curl.BUILD")), + build_file = clean_dep("//third_party:curl.BUILD"), ) tf_http_archive( @@ -421,7 +446,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): "https://github.com/antirez/linenoise/archive/c894b9e59f02203dbe4e2be657572cf88c4230c3.tar.gz", ], strip_prefix = "linenoise-c894b9e59f02203dbe4e2be657572cf88c4230c3", - build_file = str(Label("//third_party:linenoise.BUILD")), + build_file = clean_dep("//third_party:linenoise.BUILD"), ) # TODO(phawkins): currently, this rule uses an unofficial LLVM mirror. @@ -434,7 +459,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "1efbb9b05af88368be984d2f6526061d4a857181ef10f8841889a3a46869bb01", strip_prefix = "llvm-1c3cdea2f181d8e14ee184466c5fb237f1b4cda8", - build_file = str(Label("//third_party/llvm:llvm.BUILD")), + build_file = clean_dep("//third_party/llvm:llvm.BUILD"), ) tf_http_archive( @@ -445,7 +470,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "108532fb94c6f227558d45be3f3347b52539f0f58290a7bb31ec06c462d05326", strip_prefix = "lmdb-LMDB_0.9.19/libraries/liblmdb", - build_file = str(Label("//third_party:lmdb.BUILD")), + build_file = clean_dep("//third_party:lmdb.BUILD"), ) tf_http_archive( @@ -456,7 +481,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "07d34db40593d257324ec5fb9debc4dc33f29f8fb44e33a2eeb35503e61d0fe2", strip_prefix = "jsoncpp-11086dd6a7eba04289944367ca82cea71299ed70", - build_file = str(Label("//third_party:jsoncpp.BUILD")), + build_file = clean_dep("//third_party:jsoncpp.BUILD"), ) tf_http_archive( @@ -477,7 +502,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "36658cb768a54c1d4dec43c3116c27ed893e88b02ecfcb44f2166f9c0b7f2a0d", strip_prefix = "zlib-1.2.8", - build_file = str(Label("//third_party:zlib.BUILD")), + build_file = clean_dep("//third_party:zlib.BUILD"), ) tf_http_archive( @@ -487,7 +512,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): "http://www.kurims.kyoto-u.ac.jp/~ooura/fft.tgz", ], sha256 = "52bb637c70b971958ec79c9c8752b1df5ff0218a4db4510e60826e0cb79b5296", - build_file = str(Label("//third_party/fft2d:fft2d.BUILD")), + build_file = clean_dep("//third_party/fft2d:fft2d.BUILD"), ) tf_http_archive( @@ -498,7 +523,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "2f7504c73d85bac842e893340333be8cb8561710642fc9562fccdd9d2c3fcc94", strip_prefix = "snappy-1.1.4", - build_file = str(Label("//third_party:snappy.BUILD")), + build_file = clean_dep("//third_party:snappy.BUILD"), ) tf_http_archive( @@ -509,7 +534,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "2ca86fb6179ecbff789cc67c836139c1bbc0324ed8c04643405a30bf26325176", strip_prefix = "nccl-03d856977ecbaac87e598c0c4bafca96761b9ac7", - build_file = str(Label("//third_party:nccl.BUILD")), + build_file = clean_dep("//third_party:nccl.BUILD"), ) tf_http_archive( @@ -520,8 +545,8 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "dd035d57c8f19b0b612dd6eefe6e5eebad76f506e302cccb7c2066f25a83585e", strip_prefix = "librdkafka-0.11.1", - build_file = str(Label("//third_party:kafka/BUILD")), - patch_file = str(Label("//third_party/kafka:config.patch")), + build_file = clean_dep("//third_party:kafka/BUILD"), + patch_file = clean_dep("//third_party/kafka:config.patch"), ) tf_http_archive( @@ -532,7 +557,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "b888d8ce5fc10254c3dd6c9020c7764dd53cf39cf011249d0b4deda895de1b7c", strip_prefix = "aws-sdk-cpp-1.3.15", - build_file = str(Label("//third_party:aws.BUILD")), + build_file = clean_dep("//third_party:aws.BUILD"), ) java_import_external( @@ -568,7 +593,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "3c8f25c02e806c3ce0ab5fb7da1817f89fc9732709024e2a81b6b82f7cc792a8", strip_prefix = "jemalloc-4.4.0", - build_file = str(Label("//third_party:jemalloc.BUILD")), + build_file = clean_dep("//third_party:jemalloc.BUILD"), ) java_import_external( @@ -613,7 +638,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "e0928ca4aa10ea1e0551e2d7ce4d1d7ea2d84b2abbdef082b0da84268791d0c4", strip_prefix = "pprof-c0fb62ec88c411cc91194465e54db2632845b650", - build_file = str(Label("//third_party:pprof.BUILD")), + build_file = clean_dep("//third_party:pprof.BUILD"), ) tf_http_archive( @@ -624,7 +649,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): ], sha256 = "6bfa06ab52a650ae7ee6963143a0bbc667d6504822cbd9670369b598f18c58c3", strip_prefix = "cub-1.8.0", - build_file = str(Label("//third_party:cub.BUILD")), + build_file = clean_dep("//third_party:cub.BUILD"), ) tf_http_archive( @@ -635,7 +660,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): "https://github.com/cython/cython/archive/3732784c45cfb040a5b0936951d196f83a12ea17.tar.gz", ], strip_prefix = "cython-3732784c45cfb040a5b0936951d196f83a12ea17", - build_file = str(Label("//third_party:cython.BUILD")), + build_file = clean_dep("//third_party:cython.BUILD"), delete = ["BUILD.bazel"], ) @@ -657,7 +682,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): "https://mirror.bazel.build/github.com/intel/ARM_NEON_2_x86_SSE/archive/0f77d9d182265259b135dad949230ecbf1a2633d.tar.gz", "https://github.com/intel/ARM_NEON_2_x86_SSE/archive/0f77d9d182265259b135dad949230ecbf1a2633d.tar.gz", ], - build_file = str(Label("//third_party:arm_neon_2_x86_sse.BUILD")), + build_file = clean_dep("//third_party:arm_neon_2_x86_sse.BUILD"), ) tf_http_archive( @@ -668,7 +693,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): "https://mirror.bazel.build/github.com/google/flatbuffers/archive/971a68110e4fc1bace10fcb6deeb189e7e1a34ce.tar.gz", "https://github.com/google/flatbuffers/archive/971a68110e4fc1bace10fcb6deeb189e7e1a34ce.tar.gz", ], - build_file = str(Label("//third_party/flatbuffers:flatbuffers.BUILD")), + build_file = clean_dep("//third_party/flatbuffers:flatbuffers.BUILD"), ) tf_http_archive( @@ -678,7 +703,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): "https://mirror.bazel.build/storage.googleapis.com/download.tensorflow.org/models/tflite/mobilenet_v1_224_android_quant_2017_11_08.zip", "https://storage.googleapis.com/download.tensorflow.org/models/tflite/mobilenet_v1_224_android_quant_2017_11_08.zip", ], - build_file = str(Label("//third_party:tflite_mobilenet.BUILD")), + build_file = clean_dep("//third_party:tflite_mobilenet.BUILD"), ) tf_http_archive( @@ -688,7 +713,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): "https://mirror.bazel.build/storage.googleapis.com/download.tensorflow.org/models/tflite/smartreply_1.0_2017_11_01.zip", "https://storage.googleapis.com/download.tensorflow.org/models/tflite/smartreply_1.0_2017_11_01.zip" ], - build_file = str(Label("//third_party:tflite_smartreply.BUILD")), + build_file = clean_dep("//third_party:tflite_smartreply.BUILD"), ) ############################################################################## @@ -752,7 +777,7 @@ def tf_workspace(path_prefix="", tf_repo_name=""): # Needed by Protobuf native.bind( name = "python_headers", - actual = str(Label("//util/python:python_headers")), + actual = clean_dep("//util/python:python_headers"), ) # Needed by Protobuf diff --git a/third_party/mkl/BUILD b/third_party/mkl/BUILD index b27d341404..c2adf578c7 100644 --- a/third_party/mkl/BUILD +++ b/third_party/mkl/BUILD @@ -1,7 +1,5 @@ licenses(["notice"]) # 3-Clause BSD -exports_files(["LICENSE"]) - config_setting( name = "using_mkl", values = { @@ -10,17 +8,51 @@ config_setting( visibility = ["//visibility:public"], ) +config_setting( + name = "using_mkl_lnx_x64", + values = { + "cpu": "k8", + "define": "using_mkl=true", + }, + visibility = ["//visibility:public"], +) + load( "//third_party/mkl:build_defs.bzl", "if_mkl", ) +filegroup( + name = "LICENSE", + srcs = ["MKL_LICENSE"] + select({ + "@org_tensorflow//tensorflow:linux_x86_64": [ + "@mkl_linux//:LICENSE", + ], + "@org_tensorflow//tensorflow:darwin": [ + "@mkl_darwin//:LICENSE", + ], + "@org_tensorflow//tensorflow:windows": [ + "@mkl_windows//:LICENSE", + ], + }), + visibility = ["//visibility:public"], +) + cc_library( name = "intel_binary_blob", - srcs = if_mkl([ - "@mkl//:libmklml_intel.so", - "@mkl//:libiomp5.so", - ]), visibility = ["//visibility:public"], - deps = ["@mkl//:mkl_headers"], + deps = select({ + "@org_tensorflow//tensorflow:linux_x86_64": [ + "@mkl_linux//:mkl_headers", + "@mkl_linux//:mkl_libs_linux", + ], + "@org_tensorflow//tensorflow:darwin": [ + "@mkl_darwin//:mkl_headers", + "@mkl_darwin//:mkl_libs_darwin", + ], + "@org_tensorflow//tensorflow:windows": [ + "@mkl_windows//:mkl_headers", + "@mkl_windows//:mkl_libs_windows", + ], + }), ) diff --git a/third_party/mkl/MKL_LICENSE b/third_party/mkl/MKL_LICENSE new file mode 100644 index 0000000000..9c8f3ea087 --- /dev/null +++ b/third_party/mkl/MKL_LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "{}" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright {yyyy} {name of copyright owner} + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License.
\ No newline at end of file diff --git a/third_party/mkl/build_defs.bzl b/third_party/mkl/build_defs.bzl index 8b73ddabdd..53e02769da 100644 --- a/third_party/mkl/build_defs.bzl +++ b/third_party/mkl/build_defs.bzl @@ -24,6 +24,18 @@ def if_mkl(if_true, if_false = []): "//conditions:default": if_false }) +def if_mkl_lnx_x64(if_true, if_false = []): + """Shorthand for select()'ing on whether we're building with MKL. + + Returns a select statement which evaluates to if_true if we're building + with MKL enabled. Otherwise, the select statement evaluates to if_false. + + """ + return select({ + str(Label("//third_party/mkl:using_mkl_lnx_x64")): if_true, + "//conditions:default": if_false + }) + def _enable_local_mkl(repository_ctx): return _TF_MKL_ROOT in repository_ctx.os.environ diff --git a/third_party/mkl/mkl.BUILD b/third_party/mkl/mkl.BUILD index 8db97232e1..c3a71e4ff9 100644 --- a/third_party/mkl/mkl.BUILD +++ b/third_party/mkl/mkl.BUILD @@ -17,14 +17,29 @@ cc_library( visibility = ["//visibility:public"], ) -filegroup( - name = "libmklml_intel.so", - srcs = ["lib/libmklml_intel.so"], +cc_library( + name = "mkl_libs_linux", + srcs = [ + "lib/libiomp5.so", + "lib/libmklml_intel.so", + ], visibility = ["//visibility:public"], ) -filegroup( - name = "libiomp5.so", - srcs = ["lib/libiomp5.so"], +cc_library( + name = "mkl_libs_darwin", + srcs = [ + "lib/libiomp5.dylib", + "lib/libmklml.dylib", + ], + visibility = ["//visibility:public"], +) + +cc_library( + name = "mkl_libs_windows", + srcs = [ + "lib/libiomp5md.lib", + "lib/mklml.lib", + ], visibility = ["//visibility:public"], ) |