diff options
Diffstat (limited to 'tensorflow/g3doc/how_tos/hadoop/index.md')
| -rw-r--r-- | tensorflow/g3doc/how_tos/hadoop/index.md | 30 |
1 files changed, 21 insertions, 9 deletions
diff --git a/tensorflow/g3doc/how_tos/hadoop/index.md b/tensorflow/g3doc/how_tos/hadoop/index.md index a2dd67babd..2f01843604 100644 --- a/tensorflow/g3doc/how_tos/hadoop/index.md +++ b/tensorflow/g3doc/how_tos/hadoop/index.md @@ -28,26 +28,38 @@ be set: * **HADOOP_HDFS_HOME**: The location of your HDFS installation. You can also set this environment variable by running: -```shell -source ${HADOOP_HOME}/libexec/hadoop-config.sh -``` + ```shell + source ${HADOOP_HOME}/libexec/hadoop-config.sh + ``` * **LD_LIBRARY_PATH**: To include the path to libjvm.so, and optionally the path to libhdfs.so if your Hadoop distribution does not install libhdfs.so in `$HADOOP_HDFS_HOME/lib/native`. On Linux: -```shell -export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${JAVA_HOME}/jre/lib/amd64/server -``` + ```shell + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${JAVA_HOME}/jre/lib/amd64/server + ``` * **CLASSPATH**: The Hadoop jars must be added prior to running your TensorFlow program. The CLASSPATH set by `${HADOOP_HOME}/libexec/hadoop-config.sh` is insufficient. Globs must be expanded as described in the libhdfs documentation: -```shell -CLASSPATH=$($HADOOP_HDFS_HOME}/bin/hadoop classpath --glob) python your_script.py -``` + ```shell + CLASSPATH=$($HADOOP_HDFS_HOME}/bin/hadoop classpath --glob) python your_script.py + ``` + For older version of Hadoop/libhdfs (older than 2.6.0), you have to expand the + classpath wildcard manually. For more details, see + [HADOOP-10903](https://issues.apache.org/jira/browse/HADOOP-10903). + +If the Hadoop cluster is in secure mode, the following environment variable must +be set: + +* **KERB_TICKET_CACHE_PATH**: The path of Kerberos ticket cache file. For example: + + ```shell + export KERB_TICKET_CACHE_PATH=/tmp/krb5cc_10002 + ``` If you are running [Distributed TensorFlow](../distributed/index.md), then all workers must have the environment variables set and Hadoop installed. |