From 34daf66ddcf3024a0fd4f534936b152c1007dd83 Mon Sep 17 00:00:00 2001 From: kpayson64 Date: Thu, 10 May 2018 11:41:02 -0700 Subject: Clean up doc --- doc/fork_support.md | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) (limited to 'doc') diff --git a/doc/fork_support.md b/doc/fork_support.md index 141f87a435..45466434ac 100644 --- a/doc/fork_support.md +++ b/doc/fork_support.md @@ -1,11 +1,13 @@ # Background # -In gRPC python, multithreading is not usable due to GIL -(global interpreter lock).Users are using multiprocessing and -concurrent.futures module to accomplish multiprocessing. These modules fork -processes underneath. Various issues have been reported when using these -modules. Historically, we didn't support forking in gRPC, but some users seem -to be doing fine until their code started to break 1.6. This was +In Python multithreading is ineffective at concurrency for CPU bound tasks +due to the GIL. +(global interpreter lock). Users use multiprocessing, subprocess, +and concurrent.futures.ProcessPoolExecutor, to work around the GIL. +These modules call fork() underneath the hood. Various issues have +been reported when using these modules with gRPC Python. +Historically, we didn't support forking in gRPC, but some users seem +to be doing fine until their code started to break on version 1.6. This was likely caused by the addition of background c-threads and a background Python thread. @@ -19,28 +21,29 @@ forking(). ## 1.9 ## A regression was noted in cases where users are doing fork/exec. This was due to pthread_atfork() handler that was added in 1.7 to partially -support forking in gRPC. A deadlock can happen around GIL when pthread_atfork -handler is holding the lock while another thread is blocked on it and the -handler is waiting for that thread to terminate. We have provided a workaround -for this issue by allowing users to turn off the handler using env flag -```GRPC_ENABLE_FORK_SUPPORT=False```. This should be set whenever a user expects -to always call exec immediately following fork. It will disable the fork -handlers. +support forking in gRPC. A deadlock can happen when pthread_atfork +handler is running, and an application thread is calling into gRPC. +We have provided a workaround for this issue by allowing users to turn +off the handler using env flag ```GRPC_ENABLE_FORK_SUPPORT=False```. +This should be set whenever a user expects to always call exec +immediately following fork. It will disable the fork handlers. -# Future Work # ## 1.11 ## The background Python thread was removed entirely. This allows forking after creating a channel. However, the channel cannot be used by both the -parent and child process after the fork. +parent and child process after the fork. Additionaly, the process should +not fork if there are any active RPCs on the channel. -Additionally, the fork/exec workaround of setting +# Future Work # +## 1.13 ## +The workaround when using fork/exec by setting ```GRPC_ENABLE_FORK_SUPPORT=False``` should no longer be needed. Now the fork handlers are automatically not run when multiple threads are calling into gRPC. ## 1.1x ## -We would like to support forking() and using the channel from both the parent +We would like to support forking and using the channel from both the parent and child process. Additionally, we would like to support servers that use a prefork model, where the child processes accept the connections and handle requests. -- cgit v1.2.3