aboutsummaryrefslogtreecommitdiffhomepage
path: root/tensorflow/python/training
diff options
context:
space:
mode:
authorGravatar Mustafa Ispir <ispir@google.com>2018-10-03 13:33:12 -0700
committerGravatar TensorFlower Gardener <gardener@tensorflow.org>2018-10-03 13:42:27 -0700
commitd66aac16855ddb70c8d3d5b4c9d4da24a34dffec (patch)
tree03ea1b11d8212c0bdc1466652b446f528ba37dc7 /tensorflow/python/training
parent261b6958fb95db18cd28c1aba140a627deb790a1 (diff)
Updates the doc of SyncReplicasOptimizer. It notes that some worker can consume multiple mini-batches while some may not even one.
PiperOrigin-RevId: 215617588
Diffstat (limited to 'tensorflow/python/training')
-rw-r--r--tensorflow/python/training/sync_replicas_optimizer.py6
1 files changed, 5 insertions, 1 deletions
diff --git a/tensorflow/python/training/sync_replicas_optimizer.py b/tensorflow/python/training/sync_replicas_optimizer.py
index 7afaa92699..6a3756fba9 100644
--- a/tensorflow/python/training/sync_replicas_optimizer.py
+++ b/tensorflow/python/training/sync_replicas_optimizer.py
@@ -78,7 +78,11 @@ class SyncReplicasOptimizer(optimizer.Optimizer):
4. Only after all variables have been updated, increment the global step.
5. Only after step 4, pushes `global_step` in the `token_queue`, once for
each worker replica. The workers can now fetch the global step, use it to
- update its local_step variable and start the next batch.
+ update its local_step variable and start the next batch. Please note that
+ some workers can consume multiple minibatches, while some may not consume
+ even one. This is because each worker fetches minibatches as long as
+ a token exists. If one worker is stuck for some reason and does not
+ consume a token, another worker can use it.
For the replicas: