path: root/tensorflow/contrib/eager/python/examples/notebooks/1_basics.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "U9i2Dsh-ziXr"
      },
      "source": [
        "# An introduction to TensorFlow\n",
        "\n",
        "This is an introductory tutorial for using TensorFlow. It will cover:\n",
        "\n",
        "* Importing required packages\n",
        "* Creating and using Tensors\n",
        "* Using GPU acceleration\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "z1JcS5iBXMRO"
      },
      "source": [
        "## Import TensorFlow\n",
        "\n",
        "To get started, import the `tensorflow` module and enable eager execution.\n",
        "Eager execution enables a more interactive frontend to TensorFlow, the details of which we will discuss much later."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "RlIWhyeLoYnG"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "tf.enable_eager_execution()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "H9UySOPLXdaw"
      },
      "source": [
        "## Tensors\n",
        "\n",
        "A Tensor is a multi-dimensional array. Similar to NumPy `ndarray` objects, `Tensor` objects have a data type and a shape. Additionally, Tensors can reside in accelerator (like GPU) memory. TensorFlow offers a rich library of operations ([tf.add](https://www.tensorflow.org/api_docs/python/tf/add), [tf.matmul](https://www.tensorflow.org/api_docs/python/tf/matmul), [tf.linalg.inv](https://www.tensorflow.org/api_docs/python/tf/linalg/inv) etc.) that consume and produce Tensors. These operations automatically convert native Python types. For example:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "height": 125
        },
        "colab_type": "code",
        "executionInfo": {
          "elapsed": 320,
          "status": "ok",
          "timestamp": 1526420535530,
          "user": {
            "displayName": "",
            "photoUrl": "",
            "userId": ""
          },
          "user_tz": 420
        },
        "id": "ngUe237Wt48W",
        "outputId": "b1a1cd60-4eb3-443d-cd6b-68406390784e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "tf.Tensor(3, shape=(), dtype=int32)\n",
            "tf.Tensor([4 6], shape=(2,), dtype=int32)\n",
            "tf.Tensor(25, shape=(), dtype=int32)\n",
            "tf.Tensor(6, shape=(), dtype=int32)\n",
            "tf.Tensor(aGVsbG8gd29ybGQ, shape=(), dtype=string)\n",
            "tf.Tensor(13, shape=(), dtype=int32)\n"
          ]
        }
      ],
      "source": [
        "print(tf.add(1, 2))\n",
        "print(tf.add([1, 2], [3, 4]))\n",
        "print(tf.square(5))\n",
        "print(tf.reduce_sum([1, 2, 3]))\n",
        "print(tf.encode_base64(\"hello world\"))\n",
        "\n",
        "# Operator overloading is also supported\n",
        "print(tf.square(2) + tf.square(3))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "IDY4WsYRhP81"
      },
      "source": [
        "Each Tensor has a shape and a datatype"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "height": 53
        },
        "colab_type": "code",
        "executionInfo": {
          "elapsed": 215,
          "status": "ok",
          "timestamp": 1526420538162,
          "user": {
            "displayName": "",
            "photoUrl": "",
            "userId": ""
          },
          "user_tz": 420
        },
        "id": "srYWH1MdJNG7",
        "outputId": "5e4ac41c-5115-4e50-eba0-42e249c16561"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "(1, 2)\n",
            "\u003cdtype: 'int32'\u003e\n"
          ]
        }
      ],
      "source": [
        "x = tf.matmul([[1]], [[2, 3]])\n",
        "print(x.shape)\n",
        "print(x.dtype)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "eBPw8e8vrsom"
      },
      "source": [
        "The most obvious differences between NumPy arrays and TensorFlow Tensors are:\n",
        "\n",
        "1. Tensors can be backed by accelerator memory (like GPU, TPU).\n",
        "2. Tensors are immutable."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Dwi1tdW3JBw6"
      },
      "source": [
        "### NumPy Compatibility\n",
        "\n",
        "Conversion between TensorFlow Tensors and NumPy ndarrays is quite simple as:\n",
        "* TensorFlow operations automatically convert NumPy ndarrays to Tensors.\n",
        "* NumPy operations automatically convert Tensors to NumPy ndarrays.\n",
        "\n",
        "Tensors can be explicitly converted to NumPy ndarrays by invoking the `.numpy()` method on them.\n",
        "These conversions are typically cheap as the array and Tensor share the underlying memory representation if possible. However, sharing the underlying representation isn't always possible since the Tensor may be hosted in GPU memory while NumPy arrays are always backed by host memory, and the conversion will thus involve a copy from GPU to host memory."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "height": 251
        },
        "colab_type": "code",
        "executionInfo": {
          "elapsed": 238,
          "status": "ok",
          "timestamp": 1526420540562,
          "user": {
            "displayName": "",
            "photoUrl": "",
            "userId": ""
          },
          "user_tz": 420
        },
        "id": "lCUWzso6mbqR",
        "outputId": "fd0a22bc-8249-49dd-fcbd-63161cc47e46"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "TensorFlow operations convert numpy arrays to Tensors automatically\n",
            "tf.Tensor(\n",
            "[[ 42.  42.  42.]\n",
            " [ 42.  42.  42.]\n",
            " [ 42.  42.  42.]], shape=(3, 3), dtype=float64)\n",
            "And NumPy operations convert Tensors to numpy arrays automatically\n",
            "[[ 43.  43.  43.]\n",
            " [ 43.  43.  43.]\n",
            " [ 43.  43.  43.]]\n",
            "The .numpy() method explicitly converts a Tensor to a numpy array\n",
            "[[ 42.  42.  42.]\n",
            " [ 42.  42.  42.]\n",
            " [ 42.  42.  42.]]\n"
          ]
        }
      ],
      "source": [
        "import numpy as np\n",
        "\n",
        "ndarray = np.ones([3, 3])\n",
        "\n",
        "print(\"TensorFlow operations convert numpy arrays to Tensors automatically\")\n",
        "tensor = tf.multiply(ndarray, 42)\n",
        "print(tensor)\n",
        "\n",
        "\n",
        "print(\"And NumPy operations convert Tensors to numpy arrays automatically\")\n",
        "print(np.add(tensor, 1))\n",
        "\n",
        "print(\"The .numpy() method explicitly converts a Tensor to a numpy array\")\n",
        "print(tensor.numpy())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "PBNP8yTRfu_X"
      },
      "source": [
        "## GPU acceleration\n",
        "\n",
        "Many TensorFlow operations can be accelerated by using the GPU for computation. Without any annotations, TensorFlow automatically decides whether to use the GPU or CPU for an operation (and copies the tensor between CPU and GPU memory if necessary). Tensors produced by an operation are typically backed by the memory of the device on which the operation executed. For example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "height": 53
        },
        "colab_type": "code",
        "executionInfo": {
          "elapsed": 340,
          "status": "ok",
          "timestamp": 1526420543562,
          "user": {
            "displayName": "",
            "photoUrl": "",
            "userId": ""
          },
          "user_tz": 420
        },
        "id": "3Twf_Rw-gQFM",
        "outputId": "2239ae2b-adf3-4895-b1f3-464cf5361d1b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Is there a GPU available:  False\n",
            "Is the Tensor on GPU #0:   False\n"
          ]
        }
      ],
      "source": [
        "x = tf.random_uniform([3, 3])\n",
        "\n",
        "print(\"Is there a GPU available: \"),\n",
        "print(tf.test.is_gpu_available())\n",
        "\n",
        "print(\"Is the Tensor on GPU #0:  \"),\n",
        "print(x.device.endswith('GPU:0'))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "vpgYzgVXW2Ud"
      },
      "source": [
        "### Device Names\n",
        "\n",
        "The `Tensor.device` property provides a fully qualified string name of the device hosting the contents of the Tensor. This name encodes a bunch of details, such as an identifier of the network address of the host on which this program is executing and the device within that host. This is required for distributed execution of TensorFlow programs, but we'll skip that for now. The string will end with `GPU:\u003cN\u003e` if the tensor is placed on the `N`-th tensor on the host."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ZWZQCimzuqyP"
      },
      "source": [
        "\n",
        "\n",
        "### Explicit Device Placement\n",
        "\n",
        "The term \"placement\" in TensorFlow refers to how individual operations are assigned (placed on) a device for execution. As mentioned above, when there is no explicit guidance provided, TensorFlow automatically decides which device to execute an operation, and copies Tensors to that device if needed. However, TensorFlow operations can be explicitly placed on specific devices using the `tf.device` context manager. For example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "height": 53
        },
        "colab_type": "code",
        "executionInfo": {
          "elapsed": 1762,
          "status": "ok",
          "timestamp": 1526420547562,
          "user": {
            "displayName": "",
            "photoUrl": "",
            "userId": ""
          },
          "user_tz": 420
        },
        "id": "RjkNZTuauy-Q",
        "outputId": "2e613293-ccac-4db2-b793-8ceb5b5adcfd"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "On CPU:\n",
            "10 loops, best of 3: 35.8 ms per loop\n"
          ]
        }
      ],
      "source": [
        "def time_matmul(x):\n",
        "  %timeit tf.matmul(x, x)\n",
        "\n",
        "# Force execution on CPU\n",
        "print(\"On CPU:\")\n",
        "with tf.device(\"CPU:0\"):\n",
        "  x = tf.random_uniform([1000, 1000])\n",
        "  assert x.device.endswith(\"CPU:0\")\n",
        "  time_matmul(x)\n",
        "\n",
        "# Force execution on GPU #0 if available\n",
        "if tf.test.is_gpu_available():\n",
        "  with tf.device(\"GPU:0\"): # Or GPU:1 for the 2nd GPU, GPU:2 for the 3rd etc.\n",
        "    x = tf.random_uniform([1000, 1000])\n",
        "    assert x.device.endswith(\"GPU:0\")\n",
        "    time_matmul(x)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "YEOJTNiOvnpQ"
      },
      "source": [
        "## Next Steps\n",
        "\n",
        "In this tutorial we covered the most fundamental concepts in TensorFlow - `Tensor`s, operations, and devices.\n",
        "In [the next tutorial](https://github.com/tensorflow/models/tree/master/official/contrib/eager/python/examples/notebooks/2_gradients.ipynb) we will cover automatic differentiation - a building block required for training many machine learning models like neural networks."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "default_view": {},
      "name": "TensorFlow: An introduction",
      "provenance": [],
      "version": "0.3.2",
      "views": {}
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}