aboutsummaryrefslogtreecommitdiffhomepage
path: root/RELEASE.md
diff options
context:
space:
mode:
authorGravatar Amit Patankar <amitpatankar@google.com>2018-02-01 10:27:29 -0800
committerGravatar GitHub <noreply@github.com>2018-02-01 10:27:29 -0800
commit68c52b926eb8cba2b43a37cde4a9658654427e70 (patch)
tree8b3b1c5d09e3cbaee9814ae55b7d36131e77d5c9 /RELEASE.md
parent3e932b668daf1534679811c58b83478873bb9573 (diff)
Adding the known bugs to 1.5 as well.
Diffstat (limited to 'RELEASE.md')
-rw-r--r--RELEASE.md21
1 files changed, 21 insertions, 0 deletions
diff --git a/RELEASE.md b/RELEASE.md
index 728a840002..0fad3b5d41 100644
--- a/RELEASE.md
+++ b/RELEASE.md
@@ -214,6 +214,27 @@ Yoni Tsafir, yordun, Yuan (Terry) Tang, Yuxin Wu, zhengdi, Zhengsheng Wei, ç”°ä¼
* Minor refactor: move stats files from `stochastic` to `common` and remove
`stochastic`.
+## Known Bugs
+* Using XLA:GPU with CUDA 9 and CUDA 9.1 results in garbage results and/or
+ `CUDA_ILLEGAL_ADDRESS` failures.
+
+ Google discovered in mid-December 2017 that the PTX-to-SASS compiler in CUDA 9
+ and CUDA 9.1 sometimes does not properly compute the carry bit when
+ decomposing 64-bit address calculations with large offsets (e.g. `load [x +
+ large_constant]`) into 32-bit arithmetic in SASS.
+
+ As a result, these versions of `ptxas` miscompile most XLA programs which use
+ more than 4GB of temp memory. This results in garbage results and/or
+ `CUDA_ERROR_ILLEGAL_ADDRESS` failures.
+
+ A fix in CUDA 9.1.121 is expected in late February 2018. We do not expect a
+ fix for CUDA 9.0.x. Until the fix is available, the only workaround is to
+ [downgrade](https://developer.nvidia.com/cuda-toolkit-archive) to CUDA 8.0.x
+ or disable XLA:GPU.
+
+ TensorFlow will print a warning if you use XLA:GPU with a known-bad version of
+ CUDA; see e00ba24c4038e7644da417ddc639169b6ea59122.
+
## Thanks to our Contributors
This release contains contributions from many people at Google, as well as: