|
Enable collective op execution in distibuted mode:
Pass collective_graph_key into graph building and
step execution contexts (MasterSession) where it triggers
allocation of an RpcCollectiveExecutorMgr that becomes
accessible via the WorkerEnv and MasterEnv.
The collective_graph_key is used to synchronize step_ids
(which are otherwise random) between otherwise independent
graph executions that contain collective ops that need
to rendezvous.
All APIs for using collectives are still non-public and
experimental.
PiperOrigin-RevId: 199879087
|