diff options
author | 2018-10-03 13:33:12 -0700 | |
---|---|---|
committer | 2018-10-03 13:42:27 -0700 | |
commit | d66aac16855ddb70c8d3d5b4c9d4da24a34dffec (patch) | |
tree | 03ea1b11d8212c0bdc1466652b446f528ba37dc7 /tensorflow/python/training | |
parent | 261b6958fb95db18cd28c1aba140a627deb790a1 (diff) |
Updates the doc of SyncReplicasOptimizer. It notes that some worker can consume multiple mini-batches while some may not even one.
PiperOrigin-RevId: 215617588
Diffstat (limited to 'tensorflow/python/training')
-rw-r--r-- | tensorflow/python/training/sync_replicas_optimizer.py | 6 |
1 files changed, 5 insertions, 1 deletions
diff --git a/tensorflow/python/training/sync_replicas_optimizer.py b/tensorflow/python/training/sync_replicas_optimizer.py index 7afaa92699..6a3756fba9 100644 --- a/tensorflow/python/training/sync_replicas_optimizer.py +++ b/tensorflow/python/training/sync_replicas_optimizer.py @@ -78,7 +78,11 @@ class SyncReplicasOptimizer(optimizer.Optimizer): 4. Only after all variables have been updated, increment the global step. 5. Only after step 4, pushes `global_step` in the `token_queue`, once for each worker replica. The workers can now fetch the global step, use it to - update its local_step variable and start the next batch. + update its local_step variable and start the next batch. Please note that + some workers can consume multiple minibatches, while some may not consume + even one. This is because each worker fetches minibatches as long as + a token exists. If one worker is stuck for some reason and does not + consume a token, another worker can use it. For the replicas: |