diff options
Diffstat (limited to 'tensorflow/tools/docker/notebooks/2_getting_started.ipynb')
| -rw-r--r-- | tensorflow/tools/docker/notebooks/2_getting_started.ipynb | 844 |
1 files changed, 844 insertions, 0 deletions
diff --git a/tensorflow/tools/docker/notebooks/2_getting_started.ipynb b/tensorflow/tools/docker/notebooks/2_getting_started.ipynb new file mode 100644 index 0000000000..01d1c521fe --- /dev/null +++ b/tensorflow/tools/docker/notebooks/2_getting_started.ipynb @@ -0,0 +1,844 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "6TuWv0Y0sY8n" + }, + "source": [ + "# Getting Started in TensorFlow\n", + "## A look at a very simple neural network in TensorFlow" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "u9J5e2mQsYsQ" + }, + "source": [ + "This is an introduction to working with TensorFlow. It works through an example of a very simple neural network, walking through the steps of setting up the input, adding operators, setting up gradient descent, and running the computation graph. \n", + "\n", + "This tutorial presumes some familiarity with the TensorFlow computational model, which is introduced in the [Hello, TensorFlow](http://127.0.0.1:8888/tree) notebook, also available in this bundle." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "Dr2Sv0vD8rT-" + }, + "source": [ + "## A simple neural network\n", + "\n", + "Let's start with code. We're going to construct a very simple neural network computing a linear regression between two variables, y and x. The function it tries to compute is the best $w_1$ and $w_2$ it can find for the function $y = w_2 x + w_1$ for the data. The data we're going to give it is toy data, linear perturbed with random noise.\n", + "\n", + "This is what the network looks like:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAJYAAABkCAYAAABkW8nwAAAO90lEQVR4Xu2dT5Dc1J3Hv+YQT8VJ\nZUhVdprLWs4FTSrGGv4ql9CuHBCH4GaTFCLZwnIcjOAy8l6Q/1SlU4XHcg6xJgtY2OOik2KxSGoT\nGWrXzYFC2T2MDAtWitRavmQ0e9k2SYGowom4hNRPtqA9TE+rW3/cPfPepcfup6f3fu/Tv9/T+/PV\npo8//vhjsMQsULAFNjGwCrYoKy6xAAOLgVCKBRhYpZiVFcrAYgyUYgEGVilmZYUysBgDpViAgVWK\nWVmhDCzGQCkWGEuwrly5gtf++zW887/vYOn/lnD5T5cT40x9ZQrb/nEbxDtFiHeI2LJlSylGY4X2\nt8BYgUVAvfzqy3i5/TI+vPLhmq37wpYv4AHpATxw3wMMsP4cFJ5jbMAiqA4eOYg/Lv8xMcL26e34\n+vTXk8+vbv1q8n/03TsX38EfLv4h+aRE380dmmNwFY7O2gWOBVgE1Y/2/yjxUls+vwXaY1oS7tZK\n3v94MJ8zceUvV0Dea+H4AoOrQrhGHqxuT0Xjp0P7D2HqH6Yymejyu5dx5PiRZBxGnmt+bj7TdSxT\nfgv0ASuAzglwmyE8pfbZu3VaEDkDdT+AweevzGolvPjvL+LMb84knmr+yHxmqNKyCK7ZQ7OJ5yIo\n+3m6clqx8UrNB1bso2W64FQN9cnijdcdAvNAQWGRPBcLicX3Ua8S84FVcj3PnjuLhRcWkgH63OG5\nXHc7+NTBZEBP47NvffNbucpiF/e3QCaw2g0NfNvES5c+wtQ9u2G0LCj8BLAiFEaeBU0zYJ9fxkfY\njKl7FZgtCzIHIA7QUmXov/g9LmMztt6rwLBMyFROj3TkZ0fgveXh4X96GN//zvf7t2aNHGlI7VlW\n0pYmRC+AKUwAsQu5thOuvIjQEjGBGJ7CQYptdOw6etc6VzXXzcUZwJrGseWt2P28DV2I4OgyDgQK\nFgMTYtQ1xqq10eDuR6j8Fi1NxGTkwpAfRos7h05bQscQIFgibEeHMBHCVhs4EBtY8lQQd6ulvbN7\n8e6f302mC7Z/bXsuo9NkKk1X9PZ+IUyeR0sN4GscYl8DPzOP5VuPYynQwMU+dL4O3wzRbpQQ93O1\nbvQuzgRWS0p/tQA6Nuqcilq7A5u3Px28T7qw7BB1VUHqhEKTB2+pCAIVHZVD3dPgujpE6peOBzes\nQRS5nr/+b//g24nF7JN27qkCGq/J++RknHXm5JlVeiKGr/MQPQMdV0ZkCRBbNUwEMYzQhRyZEHgH\nOv29ynPM6HXtja1Rf7B4AZ7RgZv+SuMAOj+NtrYEX3avfyqMfDi2DdcLEAQBvPOX8MGtR3Ex0MEF\nJiRxP373wWZsvaeBhixDVRrg1/jxlwEWPV3ap+xVrR57Cjgpht2xEDV4mLIFvqkiaoUwwzp4U4Hv\n9/awN7YrR+vuGcAS4ZsdtKV0VNEFVqMLrIkWJGEPPP4hKA0RgiCAc1XsdJQErGQ2Ig7hOQ5sx4Hz\n0u+wvHX2akjtMWCpNhQCiCicq+AcCx1Fh9B2IegcNN6B4Teg1z0EeknzKqPFRe7a9AeLm4ajXvzU\noJEDqUahMESrKxSqbQHbDBGLoXUNlBiuUsNOT8fFQEVsNdHmdOjStTgSGOCnLTQuBDBosLxKqnTw\nntw/glPnoHMS4E6iFVjgbBGcwUGMPAjtawP73GZf/wVkAutYtAvPezYUPoKjipBdGZ5vQOgavGte\nHbfsiXD09TZUIUbg6JD3vITlrU/iYthErPOYaQk44ZhocDF8U0HDqsEOHfQaC7/2X68lyzJVTjd0\nWiJu2XMem++7+tAxSd52+hguTe3GYtjq6V3XPyqDtbA/WLyAtqRg0rHhLceo3avCsk0kjqd7uoEL\n0FJkaC/9Hh/gS9ixS0dTCaDKHVidNhoTNN2gQP/FedAmly/t2IWm2YK2xswqDbj3antzz5oToD/9\n15/i5smbcdo8vfaDQGiC37YfEyeW4KtcMu2g1HbCrp9Dx5Fw3ZCw04ZSb0Jse6CsLH1qgZFfK0zn\nn+hpznzKHGpJRzus4YJ/AX/78G94ofUC7r777pwMxAhdE6pyAK8u78CJJZ+BtcKiIw8Wea0DTx34\nZCH5oHYwM1y0TjhnziXbaWgB+4cP/RCPPfYYtm/fjpMnT+Kmm24aDrDYhdpoQdAbaMtNSB4Da6Uh\nRx4sqnB3SCTPNbtvtu9iMoU/Wg5Kt9p0h8DTp09j3759ePrpp/H4448PB1fylOtC5jTUGVifseFY\ngJXClXou+jcN6Gk2nj7JG1Gi7TG0Hkiz7OlGP/ru6OGjq46rnnjiCSwuLibe66677hocMAZWT5uN\nDVgpXGfbZ5OtybQNZq1EE6G0NXmXtGvNwbrv+4n3uu222wYPjwys9QFW2goKjbQ4Tdth6CAFeSpK\n5J3oQMUwhynS8PjMM89AVdVs3ouBtb7Aytbrw+WiMZfnednCIwOLgTUIZml43LFjB5577rnhnx4H\nuek6yztWY6yqbb+wsJBMTwwUHquu5Ijej4GVoWMoPJ4/fz7xXkM9PWa4x3rLwsDK2KMXLlxIvBeF\nR5qe2LRpU8YrN2Y2BtaA/U7hkaYnnn322exPjwPeYz1kZ2AN2YtpeCTvdeeddw5Zyvq9jIGVo28p\nPJL3ok2NLDxeb0gGVg6w0kvT8HjixIlkHJY1lauaE8GRangwsvD/noKqt+kzsLJSkCEfzdi/8cYb\nifdaKzxWoppDmxJ5FT54NH06YZShAQVmYWAVaEwqKg2PMzMzyfTEyqfHqlRzAoOH6OqwJnXoNQeB\nSWcjq0sMrJJsferUqSQsdofHylRzYg8aLyG0QtiTOvhGhFZglyKD0Mt8DKySwEqLpfD45ptvYn5+\nHr/+z19/sukwj2pOP72vyJXBy4BNME340Pg6AiNAu8IDkQysksGi4t9++2189wffxee++DkIO4Tc\nqjlrSw504Eg81FobYetq+KOwKDgagjVOnRdtBgZW0RZdpbw0BL73/nv4yZM/6bv7tVeVxkk1h4FV\nAVgbUTWHgVUBWGUcvCVV6EP/cuiztQ9NCNsMiIshrPSIeaK3oUNIlXQqaDMDqwIjlyEV0Fv6MoQl\nbENT/FTIhWSXOF2AF5jocei8cCswsAo36WcLLEPchO7yyr+9smrt6TQ3geQmcgcd2CQbIHoIDKGy\nuSwG1joEi06oU+jj3RAWR2HQgFiiTuxqJmRgVQBWGaGQDo78/OjPe9T+qpfSeBeeqIM3JPip4k8F\n7aVbMLAqMHSlg/dr7YkcCZxWg1Jz0G5UL7/EwKoArBuhmoNEbupBvPrRDhxf8qFVLFrCwKoArFQi\n4P3o/VwTpCmgdBi3r2oOIrQbNdwfGljytZ46r2U1n4FVlmW7yn3rrbfwvX/+XrKkMyPM5FLNIS2K\nbCrSNI8loKX48G6AxhIDq2SwaIcDgWWaJn71H78qRDWnlxbF1aaQxJILj6TRjRhm0L4hYrwMrJLA\nos1+BBXtyaLty5SKVs1Zverx1RB4dhIPPe/CVioeXF2rFAOrYLDIOxFQd9xxRwLVytSt90XfFaGa\nU3ATCimOgVWIGa8WkoY9AorA6pUIrqJVcwpsRiFFMbAKMONqYS9LsWWo5mS5bxV5GFg5rExhj8ZP\ndHBitbCXo+ixv5SBNWQXpmGPvNXtt98+ZCnr9zIG1oB9O2zYG/A2Y5+dgZWxC1nYy2goNt2Q3VA0\njqIDESzsZbcZ81hr2CoNe/T56KOPZrcqy8m2zazGAAt7+X8ZzGOtsCELe/mhohLGEqwyVFpY2CsG\nqLSUsQKrDJUWFvaKBWrswCpDpYWFvXKgKiYUxh5U/huwhd8idBqYRARX4bHTldd8Le8gTSpapYWW\nX0is47qnveTdi02I6aFOejlAbSdcOT2fF8NTOEixDTqnV6Uk0CC2GpW8hYTCyFXA72yj8XoAAzoE\n+nsxgNnrZc8DtL7bU9HJlDwqLY9855FkbY8ktS3LWlGLECbPo6UG8DUOsa+Bn5nH8q3HsRRo4GIS\nL6vDN0O0e70SdoB2rfeshYBF71Juyzzu90TcF59FIC8WJvSVvgiT9nnPH5nP/K7CtOPonYWzh2aT\nF2Fu+usmvPjLF3us7cXwdR6iZ6DjyogsAWKrhokghhG6kCMTAu9Ap7+r1l0cQwoLAote4+ugwT+I\nsxO78XrQKkTkqzsEkqeily8Nk0il5cfHfowv3/xlLBxf6Pk2sNhTwEkx7I6FqMHDlC3wTRVRK4QZ\n1sGbCnxfrfxgwjBtvtHXFAZW7OsQZo7hEm7Fkxf8nm+mH6TBlau0RG00OBWcY6Gj6BDaLgSdDn46\nMPwG9Hr15/MGsdco5S0GrDiAIU7D5M/AgIo9gY6Lng4+5wi3jIOea59wieCQzgEnAe4kWoEFzhbB\nGRzEyIPQDmBWpaoxSpQMUZdCwCLh1OlmDWcCBzJsSNzDiIyL8LR8Ur1lHE2nPeZzh+d6mooENW7Z\ncx6b7zuHTlvCJB1Nnz6GS1O7sUhKxDl/LEP00Vhekh8sUjThNUyYAdxr59dCSwSvAWbg5Xq7exkq\nLfRO6TMnz/TurNAEv20/Jk4swaf2xC6U2k7Y9XPoOBIm6crYh6UoaLodABOoSU3YlpLbQ48lQT0q\nnR+sEq1RBlj0dGmfsnPVOtB51IMmfEdGLQ7RkkSYkps8VbJ01QIjDdaNCIVZwOi4DnxOgsRRXIzh\nazwakY3gmphsljLWe56RBqv6wfvg3R0HFqS6CcHxC5kQHrwGo3nFSIN1Q1RaBuinyDchSyYmDRct\nhWPLPF22G2mwuo+k55kgHUylJRtZoa1A0kI0bAdGPRnSszQuYFE90yUdepoznzKHWtLRDmsglZY8\ncHZTE7UVCGqEpmtDScZZLK20wEh7LKpst9YBKQUf1A5mhovWCefMuU9eM9JbWnEQMAIY/DQOXLr+\nmqmHXkfIdj18YpSRByuFa6+2F1f+cgXkuWb3zfZdN6Twt/DCQuKpsgmVDQIXy9vPAmMB1krPRf9e\nryot/TpsXL4fG7BSuNa7Ssu4gNOvnmMFVtqY9azS0q/DxuX7sQRrXIy7kevJwNrIvV9i2xlYJRp3\nIxfNwNrIvV9i2xlYJRp3IxfNwNrIvV9i2xlYJRp3IxfNwNrIvV9i2xlYJRp3Ixf9d0NIelzdt4X5\nAAAAAElFTkSuQmCC\n", + "text/plain": [ + "<IPython.core.display.Image object>" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from IPython.display import Image\n", + "import base64\n", + "Image(data=base64.decodestring(\"iVBORw0KGgoAAAANSUhEUgAAAJYAAABkCAYAAABkW8nwAAAO90lEQVR4Xu2dT5Dc1J3Hv+YQT8VJZUhVdprLWs4FTSrGGv4ql9CuHBCH4GaTFCLZwnIcjOAy8l6Q/1SlU4XHcg6xJgtY2OOik2KxSGoTGWrXzYFC2T2MDAtWitRavmQ0e9k2SYGowom4hNRPtqA9TE+rW3/cPfPepcfup6f3fu/Tv9/T+/PVpo8//vhjsMQsULAFNjGwCrYoKy6xAAOLgVCKBRhYpZiVFcrAYgyUYgEGVilmZYUysBgDpViAgVWKWVmhDCzGQCkWGEuwrly5gtf++zW887/vYOn/lnD5T5cT40x9ZQrb/nEbxDtFiHeI2LJlSylGY4X2t8BYgUVAvfzqy3i5/TI+vPLhmq37wpYv4AHpATxw3wMMsP4cFJ5jbMAiqA4eOYg/Lv8xMcL26e34+vTXk8+vbv1q8n/03TsX38EfLv4h+aRE380dmmNwFY7O2gWOBVgE1Y/2/yjxUls+vwXaY1oS7tZK3v94MJ8zceUvV0Dea+H4AoOrQrhGHqxuT0Xjp0P7D2HqH6Yymejyu5dx5PiRZBxGnmt+bj7TdSxTfgv0ASuAzglwmyE8pfbZu3VaEDkDdT+AweevzGolvPjvL+LMb84knmr+yHxmqNKyCK7ZQ7OJ5yIo+3m6clqx8UrNB1bso2W64FQN9cnijdcdAvNAQWGRPBcLicX3Ua8S84FVcj3PnjuLhRcWkgH63OG5XHc7+NTBZEBP47NvffNbucpiF/e3QCaw2g0NfNvES5c+wtQ9u2G0LCj8BLAiFEaeBU0zYJ9fxkfYjKl7FZgtCzIHIA7QUmXov/g9LmMztt6rwLBMyFROj3TkZ0fgveXh4X96GN//zvf7t2aNHGlI7VlW0pYmRC+AKUwAsQu5thOuvIjQEjGBGJ7CQYptdOw6etc6VzXXzcUZwJrGseWt2P28DV2I4OgyDgQKFgMTYtQ1xqq10eDuR6j8Fi1NxGTkwpAfRos7h05bQscQIFgibEeHMBHCVhs4EBtY8lQQd6ulvbN78e6f302mC7Z/bXsuo9NkKk1X9PZ+IUyeR0sN4GscYl8DPzOP5VuPYynQwMU+dL4O3wzRbpQQ93O1bvQuzgRWS0p/tQA6Nuqcilq7A5u3Px28T7qw7BB1VUHqhEKTB2+pCAIVHZVD3dPgujpE6peOBzesQRS5nr/+b//g24nF7JN27qkCGq/J++RknHXm5JlVeiKGr/MQPQMdV0ZkCRBbNUwEMYzQhRyZEHgHOv29ynPM6HXtja1Rf7B4AZ7RgZv+SuMAOj+NtrYEX3avfyqMfDi2DdcLEAQBvPOX8MGtR3Ex0MEFJiRxP373wWZsvaeBhixDVRrg1/jxlwEWPV3ap+xVrR57Cjgpht2xEDV4mLIFvqkiaoUwwzp4U4Hv9/awN7YrR+vuGcAS4ZsdtKV0VNEFVqMLrIkWJGEPPP4hKA0RgiCAc1XsdJQErGQ2Ig7hOQ5sx4Hz0u+wvHX2akjtMWCpNhQCiCicq+AcCx1Fh9B2IegcNN6B4Teg1z0EeknzKqPFRe7a9AeLm4ajXvzUoJEDqUahMESrKxSqbQHbDBGLoXUNlBiuUsNOT8fFQEVsNdHmdOjStTgSGOCnLTQuBDBosLxKqnTwntw/glPnoHMS4E6iFVjgbBGcwUGMPAjtawP73GZf/wVkAutYtAvPezYUPoKjipBdGZ5vQOgavGteHbfsiXD09TZUIUbg6JD3vITlrU/iYthErPOYaQk44ZhocDF8U0HDqsEOHfQaC7/2X68lyzJVTjd0WiJu2XMem++7+tAxSd52+hguTe3GYtjq6V3XPyqDtbA/WLyAtqRg0rHhLceo3avCsk0kjqd7uoEL0FJkaC/9Hh/gS9ixS0dTCaDKHVidNhoTNN2gQP/FedAmly/t2IWm2YK2xswqDbj3antzz5oToD/915/i5smbcdo8vfaDQGiC37YfEyeW4KtcMu2g1HbCrp9Dx5Fw3ZCw04ZSb0Jse6CsLH1qgZFfK0znn+hpznzKHGpJRzus4YJ/AX/78G94ofUC7r777pwMxAhdE6pyAK8u78CJJZ+BtcKiIw8Wea0DTx34ZCH5oHYwM1y0TjhnziXbaWgB+4cP/RCPPfYYtm/fjpMnT+Kmm24aDrDYhdpoQdAbaMtNSB4Da6UhRx4sqnB3SCTPNbtvtu9iMoU/Wg5Kt9p0h8DTp09j3759ePrpp/H4448PB1fylOtC5jTUGVifseFYgJXClXou+jcN6Gk2nj7JG1Gi7TG0Hkiz7OlGP/ru6OGjq46rnnjiCSwuLibe66677hocMAZWT5uNDVgpXGfbZ5OtybQNZq1EE6G0NXmXtGvNwbrv+4n3uu222wYPjwys9QFW2goKjbQ4Tdth6CAFeSpK5J3oQMUwhynS8PjMM89AVdVs3ouBtb7Aytbrw+WiMZfnednCIwOLgTUIZml43LFjB5577rnhnx4Huek6yztWY6yqbb+wsJBMTwwUHquu5Ijej4GVoWMoPJ4/fz7xXkM9PWa4x3rLwsDK2KMXLlxIvBeFR5qe2LRpU8YrN2Y2BtaA/U7hkaYnnn322exPjwPeYz1kZ2AN2YtpeCTvdeeddw5Zyvq9jIGVo28pPJL3ok2NLDxeb0gGVg6w0kvT8HjixIlkHJY1lauaE8GRangwsvD/noKqt+kzsLJSkCEfzdi/8cYbifdaKzxWoppDmxJ5FT54NH06YZShAQVmYWAVaEwqKg2PMzMzyfTEyqfHqlRzAoOH6OqwJnXoNQeBSWcjq0sMrJJsferUqSQsdofHylRzYg8aLyG0QtiTOvhGhFZglyKD0Mt8DKySwEqLpfD45ptvYn5+Hr/+z19/sukwj2pOP72vyJXBy4BNME340Pg6AiNAu8IDkQysksGi4t9++2189wffxee++DkIO4TcqjlrSw504Eg81FobYetq+KOwKDgagjVOnRdtBgZW0RZdpbw0BL73/nv4yZM/6bv7tVeVxkk1h4FVAVgbUTWHgVUBWGUcvCVV6EP/cuiztQ9NCNsMiIshrPSIeaK3oUNIlXQqaDMDqwIjlyEV0Fv6MoQlbENT/FTIhWSXOF2AF5jocei8cCswsAo36WcLLEPchO7yyr+9smrt6TQ3geQmcgcd2CQbIHoIDKGyuSwG1joEi06oU+jj3RAWR2HQgFiiTuxqJmRgVQBWGaGQDo78/OjPe9T+qpfSeBeeqIM3JPip4k8F7aVbMLAqMHSlg/dr7YkcCZxWg1Jz0G5UL7/EwKoArBuhmoNEbupBvPrRDhxf8qFVLFrCwKoArFQi4P3o/VwTpCmgdBi3r2oOIrQbNdwfGljytZ46r2U1n4FVlmW7yn3rrbfwvX/+XrKkMyPM5FLNIS2KbCrSNI8loKX48G6AxhIDq2SwaIcDgWWaJn71H78qRDWnlxbF1aaQxJILj6TRjRhm0L4hYrwMrJLAos1+BBXtyaLty5SKVs1Zverx1RB4dhIPPe/CVioeXF2rFAOrYLDIOxFQd9xxRwLVytSt90XfFaGaU3ATCimOgVWIGa8WkoY9AorA6pUIrqJVcwpsRiFFMbAKMONqYS9LsWWo5mS5bxV5GFg5rExhj8ZPdHBitbCXo+ixv5SBNWQXpmGPvNXtt98+ZCnr9zIG1oB9O2zYG/A2Y5+dgZWxC1nYy2goNt2Q3VA0jqIDESzsZbcZ81hr2CoNe/T56KOPZrcqy8m2zazGAAt7+X8ZzGOtsCELe/mhohLGEqwyVFpY2CsGqLSUsQKrDJUWFvaKBWrswCpDpYWFvXKgKiYUxh5U/huwhd8idBqYRARX4bHTldd8Le8gTSpapYWWX0is47qnveTdi02I6aFOejlAbSdcOT2fF8NTOEixDTqnV6Uk0CC2GpW8hYTCyFXA72yj8XoAAzoE+nsxgNnrZc8DtL7bU9HJlDwqLY9855FkbY8ktS3LWlGLECbPo6UG8DUOsa+Bn5nH8q3HsRRo4GISL6vDN0O0e70SdoB2rfeshYBF71Juyzzu90TcF59FIC8WJvSVvgiT9nnPH5nP/K7CtOPonYWzh2aTF2Fu+usmvPjLF3us7cXwdR6iZ6DjyogsAWKrhokghhG6kCMTAu9Ap7+r1l0cQwoLAote4+ugwT+IsxO78XrQKkTkqzsEkqeily8Nk0il5cfHfowv3/xlLBxf6Pk2sNhTwEkx7I6FqMHDlC3wTRVRK4QZ1sGbCnxfrfxgwjBtvtHXFAZW7OsQZo7hEm7Fkxf8nm+mH6TBlau0RG00OBWcY6Gj6BDaLgSdDn46MPwG9Hr15/MGsdco5S0GrDiAIU7D5M/AgIo9gY6Lng4+5wi3jIOea59wieCQzgEnAe4kWoEFzhbBGRzEyIPQDmBWpaoxSpQMUZdCwCLh1OlmDWcCBzJsSNzDiIyL8LR8Ur1lHE2nPeZzh+d6mooENW7Zcx6b7zuHTlvCJB1Nnz6GS1O7sUhKxDl/LEP00Vhekh8sUjThNUyYAdxr59dCSwSvAWbg5Xq7exkqLfRO6TMnz/TurNAEv20/Jk4swaf2xC6U2k7Y9XPoOBIm6crYh6UoaLodABOoSU3YlpLbQ48lQT0qnR+sEq1RBlj0dGmfsnPVOtB51IMmfEdGLQ7RkkSYkps8VbJ01QIjDdaNCIVZwOi4DnxOgsRRXIzhazwakY3gmphsljLWe56RBqv6wfvg3R0HFqS6CcHxC5kQHrwGo3nFSIN1Q1RaBuinyDchSyYmDRcthWPLPF22G2mwuo+k55kgHUylJRtZoa1A0kI0bAdGPRnSszQuYFE90yUdepoznzKHWtLRDmsglZY8cHZTE7UVCGqEpmtDScZZLK20wEh7LKpst9YBKQUf1A5mhovWCefMuU9eM9JbWnEQMAIY/DQOXLr+mqmHXkfIdj18YpSRByuFa6+2F1f+cgXkuWb3zfZdN6Twt/DCQuKpsgmVDQIXy9vPAmMB1krPRf9eryot/TpsXL4fG7BSuNa7Ssu4gNOvnmMFVtqY9azS0q/DxuX7sQRrXIy7kevJwNrIvV9i2xlYJRp3IxfNwNrIvV9i2xlYJRp3IxfNwNrIvV9i2xlYJRp3IxfNwNrIvV9i2xlYJRp3Ixf9d0NIelzdt4X5AAAAAElFTkSuQmCC\"), embed=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "fBQq_R8B8rRf" + }, + "source": [ + "Here is the TensorFlow code for this simple neural network and the results of running this code:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "cellView": "both", + "colab": { + "autoexec": { + "startup": false, + "wait_interval": 0 + }, + "output_extras": [ + { + "item_id": 1 + } + ] + }, + "colab_type": "code", + "collapsed": false, + "executionInfo": { + "elapsed": 665, + "status": "ok", + "timestamp": 1446658971218, + "user": { + "color": "#1FA15D", + "displayName": "Michael Piatek", + "isAnonymous": false, + "isMe": true, + "permissionId": "00327059602783983041", + "photoUrl": "//lh6.googleusercontent.com/-wKJwK_OPl34/AAAAAAAAAAI/AAAAAAAAAlk/Rh3u6O2Z7ns/s50-c-k-no/photo.jpg", + "sessionId": "4896c353dcc58d9f", + "userId": "106975671469698476657" + }, + "user_tz": 480 + }, + "id": "Dy8pFefa_Ho_", + "outputId": "5a95f8c8-0c32-411d-956d-bb81aeed8e50" + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlsAAAEPCAYAAAB1MgENAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYVNW1///36oFuZpFmUFBQQGVwQkVQgx0cUENEo5HM\nIRqvSuKAibZmuOD3l1wFjWg0XjVxilfjnGCIUVDpoAmIiiIWCCjz1NDMMz2s3x+nWsumm56q6lRV\nf17Pc56qOnXqnFVAb1bvvc/a5u6IiIiISGJkhR2AiIiISCZTsiUiIiKSQEq2RERERBJIyZaIiIhI\nAinZEhEREUkgJVsiIiIiCRSXZMvMHjGzEjP7KGZfBzObamYLzew1M2sfj2uJiCSSmXU3szfNLGJm\n88zs2uj+iWa2wMw+NLMXzaxdzGduNbPF0ffPDS96EUlF8erZegwYXm3fLcDr7n408CZwa5yuJSKS\nSOXAje7eHxgC/NTMjgGmAv3d/QRgMdE2zcz6AZcBfYHzgQfMzEKJXERSUlySLXd/G9hcbfdI4Ino\n8yeAi+JxLRGRRHL3de7+YfT5DmAB0M3dX3f3yuhhs4Du0ecXAs+4e7m7LyNIxAYlOWwRSWGJnLPV\n2d1LIGi8gM4JvJaISNyZWU/gBOCdam9dDrwSfd4NWBnz3uroPhERILkT5LUukIikDTNrA7wAXB/t\n4ara/0ugzN3/ElpwIpJWchJ47hIz6+LuJWbWFVhf00FmpiRMJEO4e0bMVTKzHIJE60l3nxyzfzRw\nATAs5vDVwGExr7tH99V0XrV3IhmgoW1dPHu2LLpVeRkYHX3+Q2By9Q9UcfeU2MaNGxd6DKkWS6rE\noVhSOw73jMshHgXmu/u9VTvM7DzgJuBCd98bc+zLwLfMrIWZHQH0BmbXduKw/54y5d9bc/0O6R5/\nJnyHxohLz5aZPQ0UAh3NbAUwDrgDeN7MLgeWE9ytIyKS0szsdOC7wDwz+4BgCsQvgd8DLYBp0ZsN\nZ7n7GHefb2bPAfOBMmCMN7ZFFpGMFJdky92/U8tbZ8fj/CIiyeLu/waya3irzwE+cztwe8KCEpG0\npgryMQoLC8MO4XOpEkuqxAGKpSapEoc0D5nw7y3dv0O6xw+Z8R0aysLu7TYz9biLZAAzwzNkgnyi\nqL0TSX+NaevUsyUiIiKSQEq2RERERBJIyZaIiIhIAinZEhEREUkgJVsiIiIiCaRkS0RERCSBlGyJ\niIiIJJCSLREREZEEUrIlIiIikkBKtkREREQSSMmWiIiISAIp2RIRERFJICVbIiJJpHWoRZofJVsi\nIkm0c2fYEYhIsinZEhFJopKSsCMQkWRTsiUikkTr14cdgYgkm5ItEZEkUrIl0vwo2RIRSSINI4o0\nPzlhByAijROJRJgybQoAI84ZQf/+/UOOSOpDPVsizY96tkTSUCQS4eY7b2Zm2Uxmls3k5jtvJhKJ\nhB2W1IOSLZHmR8mWSJqJRCL8fNzPWdpyKbmdc+lyTBdy++d+3sslqU3DiCLNj5ItkTRS1aO1onIF\n67PW88aCN9i8eXPYYWUUM+tuZm+aWcTM5pnZddH9HcxsqpktNLPXzKx9zGduNbPFZrbAzM490PnV\nsyXS/CjZEkkjU6ZNIbd/Lt3O7Ma+8n3kbMvh45kfUxYpY8Q5I8IOL1OUAze6e39gCPATMzsGuAV4\n3d2PBt4EbgUws37AZUBf4HzgATOz2k6uni2R5kfJlkia2V65nZIWJZze/3Q6retEjw09mHjTRE2Q\njxN3X+fuH0af7wAWAN2BkcAT0cOeAC6KPr8QeMbdy919GbAYGFTb+dWzJdL8KNkSSSPHDj6WpSVL\n6bStE3l78+jZrid33XaXEq0EMbOewAnALKCLu5dAkJABnaOHdQNWxnxsdXRfjbZuhfLyREQrIqlK\npR9E0sSSzUt4c9ub3PPNe1jwzgIARtykkg+JYmZtgBeA6919h5lVX0K6UUtKt2gxnqIiaNsWCgsL\nKSwsbGqoIpJAxcXFFBcXN+kc5iEvQW9mHnYMIqlu+Zbl3Df7PkafMJoBnQeEHU6NzAx3r3WuUjox\nsxxgCvBPd783um8BUOjuJWbWFZju7n3N7BbA3X1C9LhXgXHu/k4N5/Vjj3WefBKOPz5530dE4qcx\nbZ2GEUVS3Kptq7h/9v18/7jvp2yilYEeBeZXJVpRLwOjo89/CEyO2f8tM2thZkcAvYHZtZ24c2fN\n2xJpbjSMKJLC1m5fy72z7uVbA77F8V3VFZIMZnY68F1gnpl9QDBc+AtgAvCcmV0OLCe4AxF3n29m\nzwHzgTJgzIG66zt31h2JIs2Nki2RFFWyo4R7Zt3DN/t/k5MOPSnscJoNd/83kF3L22fX8pnbgdvr\nc/4uXdSzJdLcaBhRJAWV7ipl0qxJjDxmJIO61VpFQNKQhhFFmh/1bImkmI27NnL3zLu5oM8FnHbY\nafu9P2fOHJ599jXWrVuNWTZdunRl1KjhDBw4MIRopaE6d4ZFi8KOQkSSScmWSArZsmcLk2ZN4uwj\nz2Zoj6H7vT9nzhyuvnoSe/aczpIls4DzOeKIQ5k+fRIPPjhWCVca0DCiSPOjYUSRFLFt7zbunnk3\nQ3sMZdgRw2o85tlnXyM7ezS7dm0mN/c6cnMvYffuvmRnj+bZZ19LarzSOBpGFGl+1LMlEqJIJMKU\naVPY63tZ120dI04cwbm9DriOsaS5Ll10N6JIc6OeLZGQRCIRbr7zZt7a9xZ/2f0XZhfPpse+Hgf8\nzKhRw6moeJxWrTpQVvZ7yspepGXLBVRUPM6oUcOTFLk0RadOQc+WajmLNB+qIC/NQtWkciBlJpNP\nuGcCb+97m5IOJbTPa0+rda04rcVpFN1QdMDPpeoE+UyqIJ8oVe1d27awahW0bx92RCLSUI1p6zSM\nKBmvalJ5dvZogJSZTF7mZSwpX0LXFl054qAjWF9Sv4k8AwcODD12aZqqSfJKtkSaBw0jSsarmlRe\nUHAWBQVnpcRk8r3le9nSYwu5q3NpU9KG9QvXUxYpY8Q5I0KNS5JDk+RFmhf1bIkkWVlFGQ+8+wDH\nHnkso48azT9e/wcAI24aQf/+/UOOTpJBS/aINC9KtiTjjRo1nOnTJ1FaGrwOJpOPDSWW8spy/ve9\n/6VdXju+f/z3ybIsBgzQ4tLNjWptiTQvSrYk4w0cOJAHHxwbM0E+nPlaFZUVPPz+w+Rl5/GjE39E\nlmkUv7nSMKJI86JkS5qFsCeVV3olf5rzJ9ydK066QolWM9elCyxYEHYUIpIsavFFEqzSK3nsg8fY\nW7GXq06+ipws/Y7T3KlnS6R5UbIlkkDuzpNzn2Tb3m1cc/I1SrQEULIl0twkvOU3s2XAVqASKHP3\nQYm+pkgqcHeenvc0G3Zt4NpB15KbnRt2SJIitGSPSPOSjF+zK4FCd9+chGuJ1CqZVeTdneciz7Fq\n2yquH3w9eTl5CbuWpB/1bIk0LwlfrsfMlgInu/vGWt7Xcj2ScNWryFdUPB7XKvJVC0oDfO3sr7Ew\nayELNy7khsE30Cq3VVyukeq0XE/dqtq7ykrIz4cdO6BFi7CjEpGGSNXlehyYZmYVwMPu/sckXFPk\nS2KryAOUlgb74pFsVS0onds/GCZ8/oHnGfLVIdwx8o5mk2hJw2RlQUEBbNgA3bqFHY2IJFoykq3T\n3X2tmXUiSLoWuPvbsQeMHz/+8+eFhYUUFhYmISyR+JgybQq5/XPpckwXVmxdwY59O+i0qhOtW7QO\nO7SEKi4upri4OOww0lbVUKKSLZHMl/Bky93XRh83mNlfgUFArcmWSCIko4r8qm2rWL9zPb1ye5Fn\nmT9Hq/ovRrfddlt4waQhLdkj0nwkNNkys1ZAlrvvMLPWwLmAWmSJq/pMfE9EFfmqeVola0tYsm4J\ne/ftpXdub5gfrHMociBaskek+Uh0z1YX4K9m5tFrPeXuUxN8TWlGqk98nz59Uq0T3+NZRT4SiXDV\nr66ipP0mdufvojxnDxfvuJieXXpqQWmpF92RKNJ8JDTZcvelwAmJvIY0b4mc+H4gDz3+EJGyZVir\nw9nbch85nxm5B+dSdENRg86TzHIUUn9m9ggwAihx9+Oi+04B/gDkAmXAGHd/L/rercDlQDlwfX1+\nqVStLZHmQxXkRRph7txFVLQ5mPK2u+iwbwjZFUcyd+6iBp2jqlduxoxBzJgxiKuvnsScOXMSFLE0\n0GPA8Gr7JgK/cvcTgXHAnQBm1g+4DOgLnA88YGZ13hauni2R5kPJlqS1UaOGU1HxOKWlb1Ba+kZ0\n4nv1/yPj76AjD2afl5K3vAvla7fjn1TQ+7BjGnSO2F65goKzyM4e/Xkvl4Qresd09ULMa4H20ecH\nAaujzy8EnnH3cndfBiwmuBHogJRsiTQfWqhN0lpTJr43dghv7rq5tDs9m5739WPXlsMB6JRfwU9+\n8oNGfANJI7cA/zaz3wEGnBbd3w2YGXPc6ui+A9IwokjzoWRL0l5jJr43ZGJ9bHX4YwYdw1s73uI3\nX/8NG/tvjEnWrmlwDMkoRyFx9Qhwrbv/zcwuBR4FzmnoSapK3WzdCitXFgKF8YtQROIuHjUFE75c\nT50BaLmeZqshPUvxnkheVHQ7M2YMiplY/wZDh85mwoRbv3RcbHX47ZXbWVqylAdGPcB5p54Xl/gy\naYJ8pi3XY2Y9gL/HTJDf5u7tYt7f4u4HmdktgLv7hOj+V4Fx7v5ODef8vL3bswfatw8e657hJSKp\nIlWX6xHZT0N6lhpybLxVVYfP75nP0tKlHMmRzJ0590vJVlPii2c5Cok7i25VFpvZme7+LzM7i2Bu\nFsDLwFNmNolg+LA3MLuuk+fnQ+vWwZI9nTvHO3QRSSVKtiQUDSnZkIjyDg0ZwttZuZOlpUs5puAY\n9u7Ym5T4JFxm9jTB+F5HM1tBcPfhfxHcadgC2BN9jbvPN7PngPl8URKiXt31AwfCe+/BBRck4EuI\nSMpQsiXNUn0n1g88bSAPPfMQPenJ3h17KYuUqTp8M+Du36nlrVNrOf524PaGXmfwYJg1S8mWSKZT\nsiWhqG/P0pw5cygpWceyZb9m587FtG7dJ24Tyesawlu5dSVTt0zlrkvuYvG7wYhRTdXhNdFdGmvw\nYPj978OOQkQSTRPkJTR1TQ6PnQu1Y8dOSkvv5dxz+3L99ZcnfIhuzfY1TJo5iW8f+20GHlL3tTJp\nontjZdoE+USo3t5t2AB9+sCmTZClqociaUET5CWt1NWz9OWin9CmTWu6dp2d8ESmZEcJ9866l2/2\n/2a9Ei3QRHdpnE6doKAAFiwALacpkrmUbEmzF1tH67Shp/GPTf9g5DEjGdStziLgIk02ZEgwb0vJ\nlkjmUse1pKy6luKZM2cORUW3U1R0e6PXFKyqozWzbCb/2vcvfvTkj+iX04/TDjut7g+LxEHVJHkR\nyVzq2ZKUdaA7BuNVe2vKtCns7LaT0q3b2JC1jq6tO7H2w7Uq6i1JM3gwPPhg2FGISCIp2ZKUVttc\nqHjUtopEIrz8ysu8v2keNqgDeXYYK5avZnXf1XV/WCROjjsOli6FbdugXbu6jxeR9KNkS1JWIu/w\nqxo+XNFpDfsOLyP70620phcVS7uzs4tG1yV5cnPhxBPh3XfhrLPCjkZEEkH/q0hKqhomnDFjEDNm\nDOLqqyd9aV5WXfO56jJl2hSsn1Heq4zWLXqRn3MoWYvzOLzVlRQUdEnEVxKpleZtiWQ2JVuSkr5c\n9uEssrNHf97LBV/M5xo6dDZDh85u8Hytfb6PJeVL6NGlO/l79pCb3YH83O7k5RU3KGkTiQclWyKZ\nTcOIknYaM7wYiUR45P8eYcHiBfTp1YetvbaS/2k+XXK6kH9IPivnr6PwK635yU+uqbG4anMvWCqJ\nNWQIXH01uIOpLKxIxlEFeYmreCUm1e82rKh4nAcfDJbAqWn/ga4TiUQYM24Mn+R+grUzduzcQfec\n7txx0R0sXLYQgBHn7L8Mz4HiUMK1P1WQr9uB2rvDD4fp06FXryQHJSINogryEqp4lWOA2ss+FBXd\n3uC7EKdMm8KGThvI75bPnvw95Jfmk7Uoi4XLFlJ0Q9EB44jHXY8i9TF4MMycqWRLJBMp2ZK4iXdi\nEs8lcBxnu28nz/LIL8vHUAeMpJaqeVvf+17YkYhIvCnZkrQyatRwpk+fRGlp8Dq4C3HsfsfFLsHT\np2cfyueXU766nBbegj0L9tCjQw9GnDMibtcTaarBg+Hpp8OOQkQSQXO2JG6SNb+prnlhVTW0cvvn\n4u4sWbmE0044jZaftmTh4oX07dWXK35wRY1ztBpzPQlozlbdDtTe7d4dLEq9YQO0apXkwESk3hrT\n1inZkrhKhcRkwj0TmFk2k85Hd2bhxoVsXruZb7X8Fr8Y+4ukx9KcKNmqW13t3aBBcPfdcMYZSQxK\nRBpEE+QldPGcZ9UU7s7iTYspqyyjZ05Psi077JBE6jRkSDBvS8mWSGZRsiUZo2qe1rq161i4cSE5\n5TkcmXMkFfMrGHFT3fOzRMI2eDC8+GLYUYhIvGkYUTJC1TytnH45rK5YzcalG/l6x6/TrUu3Wmto\nSXxpGLFudbV3S5fCaafBmjUqbiqSqjSMKM1SJBLh5+N+zvLK5bTPaU9WhywGZA+gW4tuddbREkkl\nRxwBBx0E770Hp5wSdjQiEi9aG1HSWlWP1vJOy1l72Frmz59Pt73dNEdL0tbFF8NLL4UdhYjEk5It\nSVtVPVpLWy4lv08+la0qyc/JZ9GbiyiLlNWrjpZIqvnGN4J5W5pdIZI5lGxJWqrq0VpRuYJ1uev4\nrPQzTj7kZDpaRw7POpyJN03UPC1pNDN7xMxKzOyjavuvNbMFZjbPzO6I2X+rmS2OvnduU6590kmw\nZw/Mn9+Us4hIKlGyJWkntker5cCWVOyuIH9TPmsXrOWI3Udw1213KdGSpnoMGB67w8wKga8Dx7r7\nscBd0f19gcuAvsD5wANmjZ/ebhb0bmkoUSRzKNmStBLbo1WSU8KiXYs4qedJFKwvoMeGHurRkrhw\n97eBzdV2XwPc4e7l0WOiizgxEnjG3cvdfRmwGBjUlOt/4xvw17825Qwikkp0N6KEpjHV5qdMm0Ju\n/1w653Zm5fyVtCxvybrN6zii3RFKtCTRjgKGmtn/ALuBn7v7+0A3YGbMcauj+xrt9NNh1aqgFMQR\nRzTlTCKSCpRsSSiqr6M4ffqkA66jWFWwtPjtYtYfuZ6KIys489gzWTJ9STBH6zYlWpJwOUAHdx9s\nZqcAzwNHNvQk48eP//x5YWEhhYWF+x2TnQ0jRwa9Wzfe2Oh4RSQOiouLKS4ubtI5VNRUQlFUdDsz\nZgyioOAsAEpL32Do0NlMmHDrfsfefffd/OZPt2NHGwd1a8vaLWs5seuJdOrSibJImXq0UkSmFTU1\nsx7A3939uOjrV4AJ7v6v6OvFwGDgSgB3vyO6/1VgnLu/U8M5693e/fOf8Nvfwttvx+PbiEi8NKat\n05wtSWl33303N99+K1ta7WBr+30sy1nBYe0Oo92ydgzJHaJESxLJoluVvwHDAMzsKKCFu28EXgZG\nmVkLMzsC6A3MburFhw2DSATWrm3qmUQkbEq2JBSjRg2nouJxSkvfoLT0DSoqHmfUqC/d/MV9991H\n0T23UHFcOX5MOZW+E9vYhtK1myk8o5CiG4qUaElCmNnTwH+Ao8xshZn9CHgUONLM5gFPAz8AcPf5\nwHPAfOAVYEw8uuvz8uCCC2Dy5KaeSUTCpmFECc2BJshPnjyZ79zwHXafvBvv4pANbMjFFmVz0K42\nvPV8sRKtFJNpw4iJ0ND27oUX4OGHYerUBAYlIg3SmLZOyZaknEgkwsU/vJhlOcvgBCjrVBbchL8g\ni+yPcph46+3cqFnDKUfJVt0a2t7t2AGHHgrLlsHBBycuLhGpP83ZkrRXVUdri2+h8tBKyveUk7sh\nFysxchZnK9GSZqVNm2Du1pQpYUciIk2h0g+SMiKRCFeNvYpFGxeR0z4Hb+Fk7ciCT6BlSUvuuOkO\nrr322lo/35i6XSKprmqtxB/8IOxIRKSxNIwoKSESiTBm3BjeL3uffQfto5JKWi5sSe6OXAryC7jz\n13cycuTIWj9fvW7Xtm2/45RT+tClS1clXkmSisOIZtYLWOXue6PL7RwH/Nndt4QUT4Pbu61boUcP\nWLQIOndOUGAiUm8aRpS0NWXaFFa2WYn3goruFWTlZ0EL6Ny+M3994q8HTLQAnn32NbKzR1NQcBa5\nuR1YvjyXqVMHMmPGIK6+ehJz5sxJzheRVPMiUGFmvYGHgcMI7iRMG+3bw0UXwZNPhh2JiDSWki1J\nCatXr2b1hg3syXWy93WgcgvkVuRywdALGnzX4Zo1rwE/oFWr0ygoOIvs7NGfDy9Ks1MZXcvwYuA+\nd78JOCTkmBrsiivgT38CDQKIpCclWxKKOXPmcPbZX6dDjwJ6DOjBv995D1q1JGtJHrYsF/u4Nbkb\n89m6vpKiotvr7JmKrdu1a9cyYAOHHqoxF6HMzL4N/BCommaeG2I8jXLGGVBZCTNn1n2siKSehM/Z\nMrPzgHsIErtH3H1Ctfc1Z6uZmTNnDl/72g9YlzsfBkVraJVBx2XDaLf7eHbsWUDl3n2wbwd9+vwP\nABUVjx9w7cSq8z777GusW7ea999fQ9u2P6n3Z6XpUnTOVj/gamCmu/8lWuH9surtUBLjaXR7N3Ei\nfPIJPPponIMSkQZJuTpbZpYFLALOAtYA7wLfcvdPYo5RstXMXHHFDTz61wfgrDI4OgvMYYWT+4/W\nnNQnKJe9bNmvKSi4hZ49LwQOvHZiTXRnYvKlYrIVy8w6AIe5+0chxtDo9m7dOjjmGFixAtq1i3Ng\nIlJvjWnrEl36YRCw2N2XA5jZM8BI4JMDfkrSQmMTmk9XfgK50R6tbKDSYCfk5+UwdGiwpNzRR5/C\nwoWtGx3bwIEDlWAJZlYMXEjQ1r0PrDezf7t72hVr69oVvvpVePZZuPLKsKMRkYZI9JytbsDKmNer\novskzVWVWpgxY1CD7/g7/vijsINzYA+w0uETh9kwqO9XmDDhViZMuJXrrvthnWsnitRDe3ffBnyD\noOTDqcDZIcfUaFdcAY88EnYUItJQKVHUdPz48Z8/LywspLCwMLRYpH5iSy0AlJYG++rTm/T9732f\n5zY+x/ollfhMoMzomtebiRNv+/yYgQMH8uCDY2N6zjTnKtUUFxdTXFwcdhh1yTGzQ4DLgF+GHUxT\nnXceXHUVfPwxDBgQdjQiUl+JTrZWA4fHvO4e3fclscmWZLZdZbuYunUqt3zrFj6b9hkftVhM78OO\n4Sc/+cF+yZSGAlNb9V+MbrvtttoPDs//A14D/u3u75rZkcDikGNqtJwcGD066N2aNCnsaESkvhI9\nQT4bWEgwQX4tMBv4trsviDlGE+TTUPWK7fW5429P+R4mzZxEr4N78c1+38QsZedSSyOk+gT5VBCP\n9u6zz2DwYFi1CvLy4hSYiNRbyt2NCJ+XfriXL0o/3FHtfSVbaaohE+T3lu/l3nfupXu77nx7wLeV\naGWgVEy2zKw7cB9wenTXW8D17r4qpHji0t4NGwZXXw2XXRaHoESkQVIy2aozACVbGW9fxT7un30/\nBa0K+P5x31eilaFSNNmaRrA8T9ViN98Dvuvu54QUT1zau2eegQcfhNSfMieSeZRsZbh0rB1VVlHG\nA+8+QNu8tow+YTRZpkULMlWKJlsfuvsJde1LYjxxae/KyqBXL3jpJTj55DgEJiL1poWoM1hTSi2E\npbyynIfff5iWuS2VaElYNprZ98wsO7p9D9gYdlBNlZsLN9wAd90VdiQiUh8pUfpB6taUUgvJFIlE\nmDJtCpVeye5euznkkEO44sQrlGhJWC4nmLM1CXDgP8DoMAOKlx//GH77W1i2DHr2DDsaETkQ/Q8o\ncROJRLj5zpv5z77/8Nye53jx9Rc5Lf80srOyww5Nmil3X+7uF7p7J3fv7O4XAZeEHVc8tGsXJFwq\nASGS+pRspYlRo4andEX1yZMnc/EPL2b2ztl8lvUZLTu3pFf3Xrz6xqthhyZSXdot1VOb666DJ5+E\nTZvCjkREDkTDiGkizIrqdU3Mnzx5MleOv5JdbXexp+MetqzYQmFeoYYOJVWl1CT+pujWDS68EB56\nCG6t3xrtIhIC3Y0oB1Sf4qXnX3o+H3T8gMrOlWxZvgUq4aC9B3FK61OYeNNE+vfvH07wklSpeDdi\nTcxshbsfXveRCbl23Nu7efNg+HBYulRFTkWSQXcjStzFTswvKDiL7OzRn/dyRSIRJtwzgU+XfcrO\n8p3QDg49/FDyVuRx0NKDlGhJaMxsu5ltq2HbDhxaj88/YmYlZvZRDe/9zMwqzezgmH23mtliM1tg\nZufG+esc0LHHwnHHwVNPJfOqItIQSrakUWInw9tZxq6Nu6h4t4K9G/bScntL7vz1nUq0JDTu3tbd\n29WwtXX3+kyfeAzYb1JktCL9OcDymH19CRa67gucDzxgSa7ce9NNQRmIyspkXlVE6kvJlhxQ9Yn5\nO3bcw+7Ktfx83M/Zc/ge9h66l3a92nFirxPpuKQjJ2w8gT+O/yMjR45s1PXmzJlDUdHtFBXdnvJ1\nxCRzufvbwOYa3poE3FRt30jgGXcvd/dlBAtdD0pshF82bFgwhPjKK8m8qojUlybIywHFTswvLS1h\n4abNrOiczYrPVrB6zWq6dezGoB6D2Lx1M0O+N4SiG4oafa3q88OmT59U5+LWIsliZhcCK919XrWO\nq27AzJjXq6P7khgb/OIXMH48fO1rwWsRSR1KtqROAwcOZODAgUy4ZwIbywrockwXNuVsYtncZZQv\nLGfzvs2URcoYcdOIJl0nXQq3SvNjZi2BXxAMITbJ+PHjP39eWFhIYWFhU08JwCWXwB13wIsvwqWX\nxuWUIgIUFxdT3MSFSJVsSYOt2b6G7S23M7DjQNosb8OQo4Yw4qYRmqMlmawX0BOYG52P1R2YY2aD\nCHqyYu9u7B7dV6PYZCuesrLgf/4Hrr8eLroIctS6i8RF9V+KbrvttgafQz+OUqeqJXhKSkqY/8l8\nNvTcRIceGWcDAAAgAElEQVStB9N6c2vu+s1d9O/f//O5VtD4RbJHjRrO9OmTKC0NXgeFW8fG86uI\nNIRFN9z9Y6Dr52+YLQUGuvtmM3sZeMrM7iYYPuwNzA4hXs49Fw45BJ54Aq64IowIRKQmqrPVzNVV\nsLTqrsPc/rms2LqCjz/5hIPmDSPfD6F93jqeeCLI8OuqxRWveCR1pUudrfows6eBQqAjUAKMc/fH\nYt5fApzs7puir28FrgDKgOvdfWot5014ezdrFnzzm7B4MeTnJ/RSIs1SY9o6JVvNWF0FSyORCD8f\n93OWd1rOoQMP5aOVEfZ93J1Dll5K7+5FlJa+wdChwS/wM2YMiplrFeyfMEElrZuTTEq2EiVZ7d3I\nkXDmmXBjxixMJJI6VNRUGuRABUsnT57MJWMu4f0N77O2ci2zl86m7e72ZJW1CDVmEanbb38LEybA\ntm1hRyIioGRLajB58mR+fNOPWd15NeX9y9m+Zzs5G3LwDfuo/GQZLSoLvrQYdqovki3S3AwYECzh\n87vfhR2JiICGEZu1moYRb7rpQn59/69Z7aspO6qMioMraLe9Hdnzszmp00lcPupy3nvvU+DLc6o0\n10o0jFi3ZLZ3S5fCySfDggXQuXNSLinSLGjOljRYbJJ08sm9efTZR3l/w/uUH1HO9uztZJNN7rZc\nuq3vxosPvKjyDlIrJVt1S3Z7d8MNsGMH/OlPSbukSMZTsiWNVnXX4dKWS1lXsY5te7fR1tpSvqSc\n/HX5/OnOPzV6CR5pHpRs1S3Z7d3WrdCvHzz3HJx+etIuK5LRNEFeGqXqrsOlLZdScEwB5QXltM1v\nS+7SXLpZNyVaImmqfftg3tY110BZWdjRiDRf6tlq5iZPnkzRxCI27dlE2dFlVPSsYEDnAWxZuIUe\nG3pw12131WvoUHO2RD1bdQujvXMPip2edx787GdJvbRIRtIwYjMRr8QmEolwyZhL2HrMVipzKykt\nLSWnPJc2Fe3oy1E89JuH6p1oxauoqaQvJVt1C6u9W7wYhgyBDz6Aww5L+uVFMoqGEZuBqsRmxoxB\nzJgxiKuvnsScOXMada4p06aQ1S+L7MOy2XPoHiw3h8r3W1IxuydbP+3A3r1763WeA9XrEpHw9ekD\nP/1pMGFeRJJPyVaaiXdi07FTR7bt2kb5lkpsd2taVHThhD6P0abNDUqYRDLILbfARx/BK6+EHYlI\n86Nkqxk7fejprNm+hj45fchf3JKc99rQr8sE2rZtWHkHFTUVSX35+fCHPwQ9XLt2hR2NSPOiOVtp\npinzoyKRCFOmTQHgK2d+hX9s+ge9s3qzfu56Vq9ezfRXl9KmzQ0NPm9VXJog37xpzlbdUqG9+973\ngrsU//CHUMMQSVuaIN9MNCaxqaqjlds/lzIvY9GaRRRdWMSPh/24SecVqaJkq26p0N5t2QInnAD3\n3w8jRoQaikhaUrIlNaqqo7W803KOPvVoVuxdQYvNLbgo/yKKbiiq9XNKvqQhlGzVLVXau7ffhm9+\nM7g7sWvXsKMRSS+6G1H2U9WjtaJyBRt9I28ufJOW3pIu2V0O+Ll43vUoIqnljDPgyith9GiorAw7\nGpHMp2Qrw02ZNoXc/rkcNewo9pTvIWtXFhvnbqQsUsaIc2ofQ1A5B5HM9utfB0OK998fdiQimS8n\n7AAkcSKRCMVvF7PMl5EzOId+/fqx9e2t9MjqwcTbJmpRaZFmLDcXnnoKBg+Gr34Vjj027IhEMpeS\nrQwViUS46ldXsaLVGtbkrySr2OiS050eWYfWawmeUaOGM336JEpLg9dBOYexSYhcRJKlVy+46y74\n9rfhnXegdeuwIxLJTBpGzFAPPf4QH5cvpbRjCyq9G2U7WrD5nZb1rgw/cOBAHnxwLEOHzmbo0Nla\nfkckQ/3gB3DyyZq/JZJIuhsxQ31l+Lm8330x3qITFRuOxjcuod2ithx1+E0MHTqbCRNuDTtEyTC6\nG7Fuqdre7dkDw4bB8OEwblzY0Yiktsa0dRpGzEDlleWUH1MGiwyraIuXlcLCClrnHBN2aCKSgvLz\n4aWX4NRToV+/oCyEiMSPkq00VlMdrIrKCv74/h/56plD2P5ma0q3dWTT5m3klLfkoN7HaO6ViNSo\na1f429/g3HOhd2848cSwIxLJHBpGTFM1LdvzwP9ezwd8wN6KvVx98tV89OFHPPvsa5SUrMO9gq5d\nu6k4qSSMhhHrlg7t3QsvwI03wuzZKngqUhNVkG9GiopuZ8aMQRQUnAXAhtJpHPzVhxh+4VcYc8oY\ncrNzG31uVY6XxsikZMvMHgFGACXuflx030Tg68Be4DPgR+6+LfrercDlQDlwvbtPreW8adHe3XYb\n/POf8Prr0KZN2NGIpBZVkG9mdu1axqerJrB41R2sKXiG3baLa065psmJlirHi/AYMLzavqlAf3c/\nAVgM3ApgZv2Ay4C+wPnAA2aW1knnf/839O8PI0fC7t1hRyOS/pRspaFIJMKqDR+zsHQsq9v/hVXH\nPcqGrOe58rjv0SK7RZPOrcrxIuDubwObq+173d2riiPMArpHn18IPOPu5e6+jCARG5SsWBPBDB5+\nGLp0gUsugXpUixGRA1CylWaq1jr8IO8DWg/PpazDIloWbOOUASewcu3KsMMTaS4uB16JPu8GxP7w\nrY7uS2vZ2fDEE8Gdit/5DpSXhx2RSPrS3YgprPrcqby8PH4+7uesqFxB1kFZZB+aTV6XFvTY3J2D\nux4cl2uqcrzIgZnZL4Eyd/9LYz4/fvz4z58XFhZSWFgYn8ASIDcX/vIXuPhi+OEP4c9/DpIwkeak\nuLiY4uLiJp1DE+RTVPW7DXfsuIf2vTezqesmNlduZs/mPZQfUk6rva0oWF9Az3Y9mXjTgdc7rO/E\nd02Ql8bIpAnyAGbWA/h71QT56L7RwJXAMHffG913C+DuPiH6+lVgnLu/U8M507K9270bvvY16NED\n/vhHyNGv6dKM6W7EDFL9bsO5C68k77R/c+KwAUyPTGd3xW46LexE3rY8Lhh8AVf84Io6E63qpSK0\nBI/EUwYmWz0Jkq1jo6/PA34HDHX3jTHH9QOeAk4lGD6cBvSpqWFL5/Zux45g/lZeHjzzDLRqFXZE\nIuFQBfkMtX17hG0755K9tZSS3SV0OrQTFQsrODLvSO56oO5FpeHLE98BSkuDfUq2RPZnZk8DhUBH\nM1sBjAN+AbQApkVvNpzl7mPcfb6ZPQfMB8qAMWmbUR1Amzbw97/DFVfA2WcHzzt2DDsqkfSQsAny\nZjbOzFaZ2Zzodl6irpWJRo0aTkXF46xY8QhzV19BZe/llO3bw3sfv0fBxgJ67+rNXbfVL9ESkYZx\n9++4+6Hunufuh7v7Y+7ex917uPvA6DYm5vjb3b23u/etrcZWJmjRIpg0/5WvwBlnwPLlYUckkh4S\nfTfi3TEN06sJvlZGGThwIDfddCHbKiaQ1XkJR519GIeeeSjd13Wn9fzWdc7Pqq4qeSstfYPS0jei\nE9+rlxESETmwrCyYMAGuuipIuObODTsikdSXsDlbZjYO2OHuv6vjuEzscW+yyZMnUzSxiE05m9jb\nYy/e0jlzwJlk78lmSO4Qim4oavA5NfFdEinT5mwlQqa1d88+Cz/9KdxzD3z3u2FHI5IcKTVBPpps\njQa2Au8BP3P3rTUcl1GNTzxEIhEuGXMJW4/ZSnmrcjZv3Ey7snYcsveQet11KBIGJVt1y8T2bu5c\nuPRSGD4cfve7YAK9SCZL+gR5M5sGdIndBTjwS+AB4P+5u5vZb4C7gStqOk861Z1JhinTppDVLwvr\nZlS2qKTAC6j8oJLDOx3eLBIt9cClh3jUnpH0d/zx8O67MHo0nHkmPP88HHZY2FGJpJaklH6oqV5N\nzHsZ95teU024ZwIvbXqJBeULaNeqHRUrK2j/cXtefPjFZpFoqURFelLPVt0yub1zhzvvhLvvhkcf\nhQsuCDsikcRIqYWozaxrzMtvAB8n6lqZpu+pfdmwYwMDcgbQbnU78j7Mo1+XU/jzn1/O+EWhtTaj\nSHoyg5tvDuZx/eQn8KMfwZYtYUclkhoSeTfiRDP7yMw+BM4EtOZLPSwsXciM7TP4w6g/MPLgkZzd\n5mza7jqWtWtHM2PGIK6+elLGJ1wikr7OPBPmzQuKnh57LLzySt2fEcl0qiCfQj7d9CkPvvcg/3XS\nf3FUx6OA/SvJl5a+wdChs5kw4dYwQ00YDSOmLw0j1q25tXdvvhkUQS0sDCbPHxyfJVxFQpVSw4jS\nMEs3L+XB9x7kihOv+DzRCtOcOXMoKrqdoqLbk9qTNnDgQB58cCxDh85m6NDZSrRE0tiwYUEvV9u2\n0Lcv3H8/lJeHHZVI8qlnKwWs2LqC37/ze0afMJoBnQd86b0wenrUuySNoZ6tujXn9m7ePBg7Ftau\nhUmT4Nxzw45IpHFSqs5WvQNoxo0PwKptq7h31r1897jvckLXE2o8JtmlEJrb0KXEh5KtujX39s4d\nXn4ZfvazoKfrt7+F4/a7R10ktWkh6jSzdvta7p11L6MGjKo10YJgaE29SiKS7sxg5Eg47zz4wx+C\nQqhDhsB//zecUHsTKJL2NGcrJOt3rueeWfdwab9LOfnQk8MO50u0jqKIJFJeHtx4I3z2WbCo9QUX\nBEnY+++HHZlIYmgYMUkikQhTpk0B4PShpzNl0xS+ftTXOf3w00OOrGaq4i4NpWHEujWX9q6hdu+G\nP/4RJk6EXr3g2mvhoosgR2MvkoI0ZytFRSIRbr7zZnL757LP97Fo7SJ+eeEv+dFXfxR2aCJxo2Sr\nbs2hvWuKsjL461/hvvtg2TK45hq48kro1CnsyES+oNIPKWrKtCnk9s/loN4Hsa79Orp06cL6uevD\nDktEJKXk5sJll8FbbwUT6T/7DPr0gW98AyZPhn37wo5QpHGUbCVJmZcxb/08urbpSqds/ZomInIg\nJ54IjzwCy5fD174WrLnYrRv89KcwcyZUVoYdoUj9KdlKgmFfHcbiNYvJ3ZxLizUtKIuUMeKcEWGH\nJSKS8tq3D6rQ/+tfMHs2dOkCP/4xHHYYjBkDr78eDD+KpDLN2UqA2Mnww746jNe2vka7Pe3YFdmF\nmTHinBH0798/5ChF4ktztuqWie1dWBYtCuZ3vfQSfPppUEbi3HPhnHOCHjCRRNEE+RQQOxm+witY\ntHoRV11wFWPPHYuZ/h+SzKVkq26Z1t6litWr4Z//hGnTgp6uQw4Jkq5hw+C006Bjx7AjlEyiZCsF\nTLhnAjPLZtLxqI58vP5jKkoruDT/Us4989wGlVJQ6QVJN0q26pZp7V0qqqiAOXNg6tRg6HHWrGDI\n8Ywzgu3UU6F3b8jSJBppJCVbKWDCPRP4975/s+HgDbTMaUnb9W3psaEHs/61ud5rDWptQklHSrbq\nlmntXTooL4ePPoK33w7ucnz3XdiyBQYOhJNOCrbjjw/uelRdL6kPLdeTAoafNZwn/vgEeRV5dM7u\nTPn8cnZmZ5GdPTpmrUF49tnXak2enn32tQYdLyIiNcvJCRKrgQPhuuuCfaWlQbX6996DZ5+FX/0K\n1qyBo4+GAQPg2GOD50cdFRRZbdEi3O8g6U/JVhyVV5bz1q63GHXuKFosbkGWZTHiphH8+c8vhx3a\nAWnIUuTLzOwRYARQ4u7HRfd1AJ4FegDLgMvcfWv0vVuBy4Fy4Hp3nxpG3FI/BQXBhPrhMauQ7dwJ\n8+fDxx8H24wZwST8FSuge/cg8TriiC9vPXtChw7Bmo8iB6JhxDipqKzgofcfIsuyuHLglWRnZX/+\nXkOHBZM5jKghS4mXTBpGNLMzgB3An2OSrQnARnefaGZFQAd3v8XM+gFPAacA3YHXgT41NWyZ0t41\nJ/v2wdKlsHhx8LhkSfC4dGlQA6ysLEjGDjss2Lp1g65dg0n6VY+dO0Pr1krKMoXmbIWk0iv505w/\nUVZRxlUnX0VO1v4dhg3tPUpWb1NR0e3MmDEoZsjyDYYOnc2ECbcm5HqSuTIp2QIwsx7A32OSrU+A\nM929xMy6AsXufoyZ3QK4u0+IHvdPYLy7v1PDOdO+vZMv274dVq2ClSuDbc0aWLsW1q0LHteuhfXr\nwT1YdqhTpyD56tgRDj74i8eDD4aDDvpia98+eFSSlno0ZysElV7J4x8+zu6y3Yw5ZQw5WTk1JkpV\nW3019HgRSbjO7l4C4O7rzKxzdH83YGbMcauj+6QZaNsW+vYNtgPZuTNIujZsCLZNm4Jt40ZYuDB4\nvmVLsG3d+sXzPXugTRto1y64Vrt2QQLWps0XW+vW0KrVF1vLll/e8vO/eKza8vKCxxYtgue5ubpD\nM5GUbDWBu/N/H/0fW/Zs4dpB15KbnbvfsNz06ZNSelhu1KjhTJ8+idLS4HVFxeOMGjU23KBE0kOj\nuqjGjx//+fPCwkIKCwvjFI6kstatv5jr1RDl5bBjB2zbFvSibdsWJG47dnz5cdeu4Pn69cHrPXtg\n9+5gq3q+Zw/s3Rs8Vm379n2x5eYGyVeLFl88z82tecvJ+eIxdsvO3v+xti0ra//nWVkH3sy+/Fh9\nX/X3q79X0wa1v5edDXl5xRQXFzfp71/DiI3k7vzl47+wettqrjv1OvJy8oD0HJbTBHmJh2YwjLgA\nKIwZRpzu7n1rGEZ8FRinYURJJ+7B/LO9e4PHffu+eNy3L0j6ysqCrep5efmXt7KyoM5ZRcUX+6pe\nV98qK/d/HbtVVAQx1bSv+v7YfbGPscdVva6+VX332racHPj737/8Z6VhxCRxd56f/zwrtq7ghsE3\nfJ5opSsNWYrUyKJblZeB0cAE4IfA5Jj9T5nZJILhw97A7OSFKdJ0Zl/0akn8KdlqIHfnr5/8lcUb\nFzN2yFjyc/K/9L6G5UTSn5k9DRQCHc1sBTAOuAN43swuB5YDlwG4+3wzew6YD5QBY9R9JSKxNIzY\nQH9f+Hc+WPcBPxvyM1q3aF3jMRqWk+Yo04YREyHd2jsR2Z9KPyTYPxf/k3dWv8PPhvyMtnltww5H\nJKUo2apbOrV3IlKzxrR1utGznqZ9No3/rPwPYwePVaIlIiIi9aZkqx6mL51O8bJibhxyI+3z24cd\njoiIiKQRJVt1eGv5W0z9bCo3DrmRDi07hB2OiIiIpBklWwcwc+VM/rH4H4wdMpaOrTqGHY6IiIik\nISVbtXh39bv87ZO/ccPgG+jcunPdHxARERGpgZKtGsxZO4fnIs9x/eDr6dqma9jhiIiISBpTslXN\nRyUf8fS8p7nu1Os4tO2hYYcjIiIiaU7JVozI+gh/nvtnfjropxzW/rCwwxEREZEMoGQr6pPST3js\nw8e45uRr6HlQz7DDERERkQyhCvJRCzYsIDsrm6M6HhV2KCJpSRXk65Yq7Z2INJ6W6xGR0CjZqpva\nO5H0p+V6RERERFKMki0RERGRBFKyJSIiIpJASrZEREREEkjJloiIiEgCKdkSERERSSAlWyIiIiIJ\npGRLREREJIGUbImIiIgkkJItERERkQRqUrJlZpea2cdmVmFmA6u9d6uZLTazBWZ2btPCFBFJDdG2\nLWJmH5nZU2bWwsw6mNlUM1toZq+ZWfuw4xSR1NHUnq15wMXAv2J3mllf4DKgL3A+8ICZpfyaacXF\nxWGH8LlUiSVV4gDFUpNUiaO5MLMewJXAie5+HJADfBu4BXjd3Y8G3gRuDS/KxMmEf2/p/h3SPX7I\njO/QUE1Kttx9obsvBqonUiOBZ9y93N2XAYuBQU25VjKk0j+AVIklVeIAxVKTVImjGdkG7ANam1kO\n0BJYTdDmPRE95gngonDCS6xM+PeW7t8h3eOHzPgODZWoOVvdgJUxr1dH94mIpC133wz8DlhB0K5t\ndffXgS7uXhI9Zh3QObwoRSTV5NR1gJlNA7rE7gIc+KW7/z1RgYmIpBozOxIYC/QAtgLPm9l3CdrE\nWNVfi0gzZu5NbxPMbDrwM3efE319C+DuPiH6+lVgnLu/U8Nn1SiJZAh3T/m5mU1hZpcB57j7ldHX\n3wcGA8OAQncvMbOuwHR371vD59XeiWSAhrZ1dfZsNUDshV8GnjKzSQTDh72B2TV9KNMbZxHJKAuB\nX5tZPrAXOAt4F9gBjAYmAD8EJtf0YbV3Is1Tk5ItM7sIuA8oAKaY2Yfufr67zzez54D5QBkwxuPR\nhSYiEiJ3n2tmfwbeByqAD4CHgbbAc2Z2ObCc4G5sEREgTsOIIiIiIlKzlKkgb2bXRgugzjOzO1Ig\nnp+ZWaWZHRzS9SdG/zw+NLMXzaxdCDGcZ2afmNkiMytK9vVj4uhuZm9GC0nOM7PrwoolGk+Wmc0x\ns5dDjqO9mT0f/XcSMbNTQ4pjvyKfYcSRylLlZ6khzOwRMysxs49i9qVN8dba2o00+w55ZvaOmX0Q\n/R7/E92fNt8B9m8z0zD+ZWY2N/r3MDu6r0HfISWSLTMrBL4OHOvuxwJ3hRxPd+AcguGAsEwF+rv7\nCQR1ypJaJNHMsoD7geFAf+DbZnZMMmOIUQ7c6O79gSHAT0KMBeB6giHysN0LvBKdiH08sCDZAdRS\n5PNbyY4jlaXYz1JDPEYQc6x0Kt5aW7uRNt/B3fcCX3X3E4HjgGFmdjpp9B2iqreZ6RZ/JcENMCe6\ne1XN0AZ9h5RItoBrgDvcvRzA3UtDjmcScFOYAbj76+5eGX05C+ie5BAGAYvdfbm7lwHPEBRuTDp3\nX+fuH0af7yBIKkKp2xZNxC8A/hTG9WPiaAd8xd0fA4gWEN4WQijVi3y2AtaEEEcqS5mfpYZw97eB\nzdV2p03x1lraje6k0XcAcPdd0ad5BP9nbyaNvkMtbWbaxB9l7J8vNeg7pEqydRQw1Mxmmdl0Mzs5\nrEDM7EJgpbvPCyuGGlwO/DPJ16xemHYVKVCY1sx6AicA+5URSZKqRDzsyY5HAKVm9li0e/5hM2uZ\n7CBqKPK5JVrkU76Qkj9LjdQ5HYu3xrQbs0izArTRIbgPgHVAsbvPJ72+Q01tZjrFD0Hs08zsXTP7\ncXRfg75DPEs/HJDVXhz1V9E4Orj7YDM7BXgOODKkWH5BMIQY+16y4/i8YKyZ/RIoc/enExVHujCz\nNsALwPXR31STff2vASXu/mF06DvM2/hzgIHAT9z9PTO7h6Bbe1wyg7D9i3y+YGbf0b/XZiPsXzrq\nVL3dsP1rnaX0d4iOcJwY7c1+Ldr2pMV3qKHNrE1Kxh/jdHdfa2adgKlmtpAG/h0kLdly93Nqe8/M\nrgZeih73bnRiekd335jMWMxsANATmGtmRtDl/L6ZDXL39cmKIyae0QTdr8Pife16WA0cHvO6e3Rf\nKKJDVC8AT7p7jTWMkuB04EIzu4BgTby2ZvZnd/9BCLGsIuiBfS/6+gUgjInXJwP/dvdNAGb2EnAa\noGTrCyn1s9REJWbWJaZ4a9zbxXiqpd1Iq+9Qxd23mdkrBD9z6fIdamoznwTWpUn8ALj72ujjBjP7\nG8HUgAb9HaTKMOLfiCYUZnYUkJuoROtA3P1jd+/q7ke6+xEE/6GdmIhEqy5mdh5B1+uF0UmSyfYu\n0NvMekTvLvsWQbHasDwKzHf3e8MKwN1/4e6Hu/uRBH8eb4aUaBHtvl4Z/XmBoLhmGJP2FwKDzSw/\n+gvKWYQwUT/FpdrPUkMY+xesHh19Xmvx1hRSU7uRNt/BzAqq7nKLThM4h6C2W1p8h1razO8DfycN\n4gcws1bR3lHMrDVwLjCPBv4dJK1nqw6PAY+a2TyCqsyh/AdWAye8oaL7gBYE48QAs9x9TLIu7u4V\nZvZTgrsis4BH3D2U/0Sjd998F5gXnbvgwC/c/dUw4kkh1xGs1JALLAF+lOwADlDkU6JS6WepIczs\naaAQ6GhmKwiGqO8gWA8y5Yu31tZuEFT5T5cCtIcAT0R/kcki6KF7I/p90uU71OQO0if+LsBfo8PP\nOcBT7j7VzN6jAd9BRU1FREREEihVhhFFREREMpKSLREREZEEUrIlIiIikkBKtkREREQSSMmWiIiI\nSAIp2RIRERFJICVbIiKStszsYDP7ILpG6FozWxXzul61JM3sETPrU8cxY8zs2/GJusbzXxxTpFgy\njOpsiYhIRjCz/wZ2uPvdNbxnnsL/4UWXsXkhxOXIJIHUsyUiIpni8xU/zKyXmUXM7P/M7GOgq5k9\nZGazzWyemf0q5ti3zOw4M8s2s81mdruZfWhm/zazgugx/5+ZXRdz/O1m9o6ZLTCzwdH9rczsBTP7\n2MyeN7N3zey4/YI0uzMa24fR85xBsA7u3dEeucPNrLeZvRo9R7GZ9Y5+9kkze8DM3jOzT6JLu2Fm\nA6LfbU70vD0T9qcsDZYqy/WIiIjE29HA99z9AwAzK3L3LWaWDUw3sxfc/ZNqn2kPTHf3W83sd8Dl\nwMSaTu7up5rZ1wmWMjofuBZY6+6XRpOs96t/xsw6A+e7e//o63Yxi0w/7+4vR/e/CVzh7kvN7DTg\nD8Dw6Gm6u/vJ0WHH182sFzAGuNPdn48u4RXWUnNSAyVbIiKSqT6rSrSivhtdyy6HYN3BfkD1ZGuX\nu0+NPn8fOKOWc78Uc0yP6PMzCNb9w90/MrNIDZ/bBFSY2cPAK8CU6gdEF58eDLwYXRcRvjwS9Vz0\nGoui61b2Af4D/Drao/WSu39WS9wSAg0jiohIptpZ9SQ6DHcdUOjuxwOvAfk1fGZfzPMKau+U2FuP\nY/brXXL3cuBk4G/ARcA/avncBncf6O4nRrfjY09T7Vh39/+Lnm8v8Gp0aFJShJItERHJVLHJTjtg\nG7DDzA7hiyG5A32mof4NjAIws2OBvvud3KwN0N7dXwFuBE6IvrU9GiPuvgVYa2YXRT9j1eZ+fTO6\n/yigO7DYzI5w9yXu/nuC3rL95opJeDSMKCIimerzHiB3n2NmC4AFwHLg7ZqOq/a8zvNWcx/wRHRC\n/vzotrXaMe2Bl8wsjyCxGxvd/xfgITO7kaCH6lvAg2Y2HsgF/g/4KHrsajN7D2gNXOnu5Wb2nWhp\nig7DR7AAAAB7SURBVDJgNcE8MkkRKv0gIiISB9GJ9znuvjc6bPka0MfdK+N4jSeJmUgv6UE9WyIi\nIvHRBngjppjqf8Uz0YpSD0kaUs+WiIiISAJpgryIiIhIAinZEhEREUkgJVsiIiIiCaRkS0RERCSB\nlGyJiIiIJJCSLREREZEE+v8BDWQFu2iG7q0AAAAASUVORK5CYII=\n", + "text/plain": [ + "<matplotlib.figure.Figure at 0x7f804039d090>" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "#@test {\"output\": \"ignore\"}\n", + "import tensorflow as tf\n", + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "\n", + "%matplotlib inline\n", + "\n", + "# Set up the data with a noisy linear relationship between X and Y.\n", + "num_examples = 50\n", + "X = np.array([np.linspace(-2, 4, num_examples), np.linspace(-6, 6, num_examples)])\n", + "X += np.random.randn(2, num_examples)\n", + "x, y = X\n", + "x_with_bias = np.array([(1., a) for a in x]).astype(np.float32)\n", + "\n", + "losses = []\n", + "training_steps = 50\n", + "learning_rate = 0.002\n", + "\n", + "with tf.Session() as sess:\n", + " # Set up all the tensors, variables, and operations.\n", + " input = tf.constant(x_with_bias)\n", + " target = tf.constant(np.transpose([y]).astype(np.float32))\n", + " weights = tf.Variable(tf.random_normal([2, 1], 0, 0.1))\n", + " \n", + " tf.initialize_all_variables().run()\n", + " \n", + " yhat = tf.matmul(input, weights)\n", + " yerror = tf.sub(yhat, target)\n", + " loss = tf.reduce_mean(tf.nn.l2_loss(yerror))\n", + " \n", + " update_weights = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)\n", + " \n", + " for _ in range(training_steps):\n", + " # Repeatedly run the operations, updating the TensorFlow variable.\n", + " update_weights.run()\n", + " losses.append(loss.eval())\n", + "\n", + " # Training is done, get the final values for the graphs\n", + " betas = weights.eval()\n", + " yhat = yhat.eval()\n", + "\n", + "# Show the fit and the loss over time.\n", + "fig, (ax1, ax2) = plt.subplots(1, 2)\n", + "plt.subplots_adjust(wspace=.3)\n", + "fig.set_size_inches(10, 4)\n", + "ax1.scatter(x, y, alpha=.7)\n", + "ax1.scatter(x, np.transpose(yhat)[0], c=\"g\", alpha=.6)\n", + "line_x_range = (-4, 6)\n", + "ax1.plot(line_x_range, [betas[0] + a * betas[1] for a in line_x_range], \"g\", alpha=0.6)\n", + "ax2.plot(range(0, training_steps), losses)\n", + "ax2.set_ylabel(\"Loss\")\n", + "ax2.set_xlabel(\"Training steps\")\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "vNtkU8h18rOv" + }, + "source": [ + "In the remainder of this notebook, we'll go through this example in more detail." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "r6rsv-q5gnn-" + }, + "source": [ + "## From the beginning\n", + "\n", + "Let's walk through exactly what this is doing from the beginning. We'll start with what the data looks like, then we'll look at this neural network, what is executed when, what gradient descent is doing, and how it all works together." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "UgtkJKqAjuDj" + }, + "source": [ + "## The data\n", + "\n", + "This is a toy data set here. We have 50 (x,y) data points. At first, the data is perfectly linear." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "cellView": "form", + "colab": { + "autoexec": { + "startup": false, + "wait_interval": 0 + }, + "output_extras": [ + { + "item_id": 1 + } + ] + }, + "colab_type": "code", + "collapsed": false, + "executionInfo": { + "elapsed": 398, + "status": "ok", + "timestamp": 1446659128547, + "user": { + "color": "#1FA15D", + "displayName": "Michael Piatek", + "isAnonymous": false, + "isMe": true, + "permissionId": "00327059602783983041", + "photoUrl": "//lh6.googleusercontent.com/-wKJwK_OPl34/AAAAAAAAAAI/AAAAAAAAAlk/Rh3u6O2Z7ns/s50-c-k-no/photo.jpg", + "sessionId": "4896c353dcc58d9f", + "userId": "106975671469698476657" + }, + "user_tz": 480 + }, + "id": "-uoBWol3klhA", + "outputId": "efef4adf-42de-4e6f-e0c3-07ddd3083d85" + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQMAAAEACAYAAAC3RRNlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEGpJREFUeJzt3XuMHeV5x/Hv4xinBgIN2Yq0WCFFJK1EUzC3otAWC4JB\nRHESkUq0qRKIBG2jAE1cZBKoQL0kBGSRtEr/IBdE2iDUEkhMLrJNqS1BBeF+dYA2KbcALYUWIVyu\nT/8447Ase9a7O+/MOTPn+5Esn4Nn552F9Y9553nP80ZmIklLRn0BksaDYSAJMAwkVQwDSYBhIKli\nGEgCCoVBRHw2Iu6NiLsi4lsRsazEeSW1p3YYRMS+wKnAysz8TWApcFLd80pq19IC53gWeBHYLSJe\nBXYFflbgvJJaVPvOIDOfAdYDDwOPAf+TmdfWPa+kdpWYJuwHfBrYF/gVYPeI+IO655XUrhLThEOB\nGzLzaYCIuAp4L3D59IMiwg9BSCOSmbGzY0pUE+4HjoiIX4iIAI4Btg25oNZ+nXfeeY7X0fH6/L2N\nYrz5KvHM4E7gm8CtwJ1AAJfUPa+kdpWYJpCZFwEXlTiXpNHo7QrEVatWOV5Hx+vz9zaK8eYrFjKn\nqDVQRLY1lqTXRATZ0gNEST1gGEgCDANJFcNAEmAYSKoYBpIAw0BSxTCQBBgGkiqGgSTAMJBUMQwk\nAYaBpIphIAkwDCRVDANJgGEgqVJqr8U9I+KfImJbtefib5U4r6T2FGmICnwZ+EFm/l5ELGWwxZqk\nDimxo9IewO9k5qUAmflyZj5b+8qkntm4cSOrV5/I6tUnsnHjxlFfzhvUbogaEQcy2CfhPuBA4Bbg\nzMzcPuM4G6JqYm3cuJEPf/jjbN/+RQCWL1/H1VdfxnHHHdf42G02RF0KHAx8JTMPBp4Hzi5wXqk3\n1q+/pAqCjwODUFi/frz2GirxzOBR4JHMvKV6fyWwbrYDzz///J+/XrVq1dj2j5dK2bhxI+vXX8Kt\nt94JrGllzC1btrBly5YFf12RfRMiYitwamY+EBHnAbtm5roZxzhN0ER5/dTgbuCrwN8A4zlNKFVN\nOAP4VkTsAvwEOKXQeaXOev3UYGCvvf6SQw45kLVr2wmChSi11+KdwGElziV12Y5pAcBTT/33jD99\nD4cc8lM2bfp2+xc2D6XuDKSJN7NisGzZn7Js2Vm8+OLgz5cvX8fatZeN8ArnZhhIhcycFrz4Iqxc\n+VWmpjYAjOXUYDrDQGrQ1NTeYzstmMkwkGra8Zzgqaee7NS0YCa3ZJdqmO05wQEHHMjU1NtYu/a0\nsZgWtF1alCbSbM8JpqY2dGZqMJ39DKRF2PGho8HKwn7wzkBaoNdPDX6VwZq7ga49J5jOMJAWqGsr\nC+fLMJBqG++VhfNlGEjzMH2Z8VFHHcz1169je9Wxo8tTg+ksLUo7MVtjknPOOZ2tW28DGJsS4jCW\nFqVCZj4j2L4dtm7tZvlwLpYWpSH6WD6ci3cG0iz6Wj6ci2EgzaKv5cO5GAZSpcuNSUowDCS635ik\nBMNAovuNSUooFgYRsYTBBiqPZmY7PaGlBnWpMUkJJe8MzmSwq9IeBc8pNaovjUlKKBIGEbECOAH4\na+AzJc4pNW225wQrV15aNSbp/7RgplJ3BhcDZwF7Fjqf1Lg+NSYpoXYYRMT7gScz846IWAUMXQPt\n9moaB6PY8qxNI9teLSI+D/wh8DKwHHgLcFVmfmzGcX5QSSM3LluetWm+H1Qq+qnFiDgKWDtbNcEw\n0DhYvfpENm9ew2srC/+Mvfb6TrWycLw/fbhYfmpRmpf+ryycr6JhkJlbga0lzynVNQmNSUqwuYl6\nreuNSUpwmiAxOY1JSrC5iXpp0hqTlOCdgXpnEhuTlGAYqHcmsTFJCYaBemHSG5OUYBio82xMUoZh\noM6zMUkZhoF6adIak5RgGKizbExSlisQ1UmzPSc44IADq8Yk/V9VuBCuQFSv2ZikPMNAnTF3+VB1\nGQbqBMuHzTMM1AmWD5tnGKizLB+WZRhorFk+bI+lRY0ty4dlWFpU51k+bFft5iYRsSIirouIeyPi\n7og4Y+dfJQ1nY5LRKHFn8DLwmWoTld2BWyNiU2b+uMC5NWFsTDI6tcMgM58AnqhePxcR24B9AMNA\nC2ZjktEp+swgIt4JHATcVPK8mmQ2JmlLsTCopghXAmdm5nOzHeNei5qN+xqUNbK9FgEiYinwPeCH\nmfnlIcdYWtQbuK9B89ouLX4DuG9YEEjDuK/B+ChRWjwS+ChwdETcHhG3RcTx9S9NfWb5cPyUqCbc\nALypwLVoQlg+HE+uQFTrLB+OJ8NArXBfg/FnGKhxNibpBsNAjbMxSTcYBhoJG5OMH8NAjbExSbfY\n3ESNsDHJ+LC5iUbKxiTdU3sFojSdKwu7yzsDFePKwm4zDFSMKwu7zTBQg1xZ2CWGgWqxMUl/WFrU\notmYpBssLapxNibpF0uLWjDLh/3knYEWxPJhfxkGWhDLh/1lGGinbEwyGYqEQdUA9UsMnkF8PTO/\nWOK8Gj0bk0yO2qXFiFgCPAAcA/wMuBk4aeZei5YWu2n16hPZvHkNr00LLqsak+wNWD7sgjZLi4cD\nD2bmQ9XAVwAfxL0We8vGJP1UIgz2AR6Z9v5RBgGhDrMxyeRp9QGiey12w2zPCVauvLRqTGLFYNyN\nbK/FiDgCOD8zj6/enw3kzIeIPjPojtmeExx7rCsLu6rNZwY3A/tHxL7A48BJwO8XOK9aNHf5UJOg\nxPZqr0TEp4BNvFZa3Fb7ytQay4cCP7UoLB/2nZ9aVC2WDyePYTDBLB9qOqcJE8p9DSaH0wTNyX0N\nNJPNTSaMjUk0jHcGE8TGJJqLYTBBbEyiuRgGE83GJHqNYdBz7mug+bK02GPuayCwtCjc10ALY2mx\nhywfajG8M+gZy4daLMOgZywfarEMgx5wXwOVYBh0nI1JVIph0HGzfeBo0JhkA4BTA82bYdBDNibR\nYhgGHWVjEpVWawViRFwIfAB4Afh34JTMfHbIsa5ALMTGJFqI+a5ArBsG7wOuy8xXI+ICBvslfHbI\nsYZBIe5roIWYbxjUWoGYmddm5qvV2xuBFXXOp7m5slBNKvnM4BPAFQXPp2lcWaim7TQMImIzsPf0\nfwQkcE5mXlMdcw7wUmZePte53Gtx8VxZqPka5V6LJwOnAkdn5gtzHOczgxp8TqDFauUjzBFxPHAW\n8LtzBYEWx8YkalPdasKDwDJgx4L4GzPzk0OO9c5gAWxMolJauTPIzHfV+XoNZ2MStc3mJmPG8qFG\nxeXIY8TyoUbJMBgjlg81SobBiNmYROPCMBghG5NonBgGI2RjEo0Tw2DM2JhEo2IYjICNSTSO3F6t\nZTYmUdvcXm1MzfacYGrKlYUaPcOgBXOXD6XxYBg0zPKhusIwaJjlQ3WFYTAClg81jgyDhlg+VNdY\nWmyA5UONE0uLI2T5UF1kc5OCbEyiLvPOoBAbk6jrioRBRKwFLgKmMvPpEufsGhuTqOtqh0FErACO\nBR6qfzl9YmMSdUuJO4OLGeydsKHAuTrFfQ3UJ3U3UVkDPJKZd0fstHLRKzPLh9dfv2NfA1cWqpvq\n7LV4LvA5BlOE6X82VJ/2WnRfA42r1vdajIjfAK4FnmcQAiuAx4DDM/M/Zzm+F4uOdkwNbr31Tp5+\n+s9x70ONu8YXHWXmPcDbpw34U+DgzHxmseccd5YP1Wcl1xkkO5kmdJ3lQ/VZsTDIzP1KnWucuK+B\nJoUrEOdgYxJNEsNgDjYm0SQxDBbIxiTqK8NgFjYm0SSyuckMNiZR39jcZJFsTKJJZXOTio1JNOm8\nM8CVhRIYBoArCyUwDIZwZaEmz8SGgY1JpNebyNLizPLh8uU7GpPcBmAJUb1iaXEONiaR3miiSouW\nD6XhJubOwPKhNLeJCQPLh9Lceh0GNiaR5q+3YWBjEmlhSuyodDrwSeBl4PuZeXbtqyrAxiTSwtTd\nRGUV8AHgPZn5ckRMFbmqhtiYRBqu7p3BnwAXZObLAJn5VP1LqsfGJNLi1FqBGBG3A98Fjge2A2dl\n5i1Djm18BaKNSaQ3KrYCcSfbqy0F3pqZR0TEYcA/AiNrmW5jEmnxdhoGmXnssD+LiD8GrqqOuzki\nXo2It2XmzDoe0K+9FqVx1fpeiwARcRqwT2aeFxHvBjZn5r5Djm19mrB8+TquvtqqgSbbfKcJdcNg\nF+AbwEHAC8DazNw65NhWPrU4faGRzwmklsJgIcbpI8zSJJlvGEzUpxYlDWcYSAIMA0kVw0ASYBhI\nqhgGkgDDQFLFMJAEGAaSKoaBJMAwkFQxDCQBhoGkimEgCTAMJFUMA0mAYSCpYhhIAgwDSZVaYRAR\nh0XEjyLi9ur3Q0tdmKR21b0zuBA4NzNXAucBF9W/pDIW0zfe8cZjvD5/b6MYb77qhsHjwJ7V618E\nHqt5vmL6/h+4z+P1+XsbxXjzVXfj1bOBGyJiPYNt195b/5IkjULdvRZPB07PzO9ExEcYbKgydDs2\nSeOr7o5Kz2bmHtPe/29m7jnkWHdQkUakyC7MO/FgRByVmVsj4hjggToXI2l06obBHwFfiYhlwP8B\np9W/JEmj0Npei5LGW6srECPiLyLizoi4IyKujYgVDY93YURsq8b7dkTssfOvWvRYH4mIeyLilYg4\nuMFxjo+IH0fEAxGxrqlxqrG+HhFPRsRdTY4zbbwVEXFdRNwbEXdHxBkNj/fmiLipWjR3b0R8vsnx\nqjGXRMRtEbGh6bGq8f6j+jt3e0T8aM6DM7O1X8Du016fDnyt4fHeByypXl8AfKHBsX4NeBdwHXBw\nQ2MsAf4N2BfYBbgD+PUGv6ffBg4C7mrp5+PtwEE7flaA+5v8/qpxdq1+fxNwI3Bkw+N9GvgHYENL\n/05/Arx1Pse2emeQmc9Ne7sb8FTD412bma9Wb28EGrsTycz7M/NBBqXXphwOPJiZD2XmS8AVwAeb\nGiwzrweeaer8s4z3RGbeUb1+DtgG7NPwmM9XL9/MIGwb+36rO+ETgK81NcZswzLPGUDrH1SKiL+K\niIeBk4EvtDj0J4AftjheE/YBHpn2/lEa/ssyKhHxTgZ3JTc1PM6SiLgdeALYkpn3NTjcxcBZDNbp\ntCWBzRFxc0ScOteBdasJbzDHIqVzMvOazDwXOLea734JOKXJ8apjzgFeyszLmx5L9UXE7sCVwJkz\n7iaLq+4cV1bPkzbtKJWXHici3g88mZl3RMQqmr2DnO7IzHw8In6JQShsq+743qB4GGTmfFcgXg78\noOnxIuJkBrdmRzc9VgseA94x7f0KxujzICVExFIGQfD3mfndtsbNzGcj4vvAoUDxMACOBNZExAnA\ncuAtEfHNzPxYA2P9XGY+Xv3+XxFxNYOp5qxh0HY1Yf9pbz/E4AFYk+Mdz+C2bE1mvtDkWDOHbui8\nNwP7R8S+1dqOk4Cmn0oH7f1fDAZL2u/LzC83PVBETEXEntXr5QyW0jfyM5mZn8vMd2Tmfgz+u13X\ndBBExK7VXRYRsRuwGrhn2PFtPzO4ICLuquZoq4C1DY/3twyeSm+uyjl/19RAEfGhiHgEOAL4XkQU\nfz6Rma8AnwI2AfcCV2TmttLj7BARlwP/Crw7Ih6OiFpTunmMdyTwUeDoqhR2WxXoTfll4F+qn8cb\nGTzh/+cGx2vb3sD1076/azJz07CDXXQkCbDtmaSKYSAJMAwkVQwDSYBhIKliGEgCDANJFcNAEgD/\nDzcvNav5fpAxAAAAAElFTkSuQmCC\n", + "text/plain": [ + "<matplotlib.figure.Figure at 0x7f8030785b10>" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "#@test {\"output\": \"ignore\"}\n", + "num_examples = 50\n", + "X = np.array([np.linspace(-2, 4, num_examples), np.linspace(-6, 6, num_examples)])\n", + "plt.figure(figsize=(4,4))\n", + "plt.scatter(X[0], X[1])\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "AId3xHBNlcnk" + }, + "source": [ + "Then we perturb it with noise:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "cellView": "form", + "colab": { + "autoexec": { + "startup": false, + "wait_interval": 0 + }, + "output_extras": [ + { + "item_id": 1 + } + ] + }, + "colab_type": "code", + "collapsed": false, + "executionInfo": { + "elapsed": 327, + "status": "ok", + "timestamp": 1446659134929, + "user": { + "color": "#1FA15D", + "displayName": "Michael Piatek", + "isAnonymous": false, + "isMe": true, + "permissionId": "00327059602783983041", + "photoUrl": "//lh6.googleusercontent.com/-wKJwK_OPl34/AAAAAAAAAAI/AAAAAAAAAlk/Rh3u6O2Z7ns/s50-c-k-no/photo.jpg", + "sessionId": "4896c353dcc58d9f", + "userId": "106975671469698476657" + }, + "user_tz": 480 + }, + "id": "fXcGNNtjlX63", + "outputId": "231c945e-e4a4-409e-b75b-8a8fe1fdfc30" + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQkAAAEACAYAAACgZ4OsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEptJREFUeJzt3X+MpVV9x/H3B5dJx6rYFbNGNqjUX2S1wFbRipWNZHYo\nhtUV02B/iJq4bbBI64QssCZLSmyKdKOGYixRYW3cUH/Rbi3MMgZ3G0ypRUC3gLCt1gKKBrES063L\nj2//uHfWu8OdZ+7Mvec553nu55Xc7Nx7n5nnTLL3O+d8z/eco4jAzGwxR+VugJmVzUHCzCo5SJhZ\nJQcJM6vkIGFmlRwkzKxS8iAh6RJJd0v6tqTPSZpIfU8zG52kQULSi4D3AadExG8Aq4BzU97TzEZr\nVeKf/xhwCPhVSU8BzwR+kPieZjZCSXsSEfFTYAfw38BDwP9ExFdT3tPMRiv1cOME4M+AFwEvBJ4l\n6fdS3tPMRiv1cOM1wNcj4lEASV8G3gDsmr9AkhePmGUUEap6P/Xsxn3A6yX9iiQBZwD3LrwoIhr1\n2L59e/Y2tL3NTWtvU9s8iNQ5iW8BnwW+CXwLEHBNynua2WilHm4QEVcCV6a+j5ml4YrLFdiwYUPu\nJixb09rctPZCM9s8CA06LknWAClyt8FsXEkiMicuzazhHCTMrJKDhJlVcpAws0oOEmZWyUHCzCo5\nSJhZJQcJM6vkIGFmlRwkzKySg4SZVXKQMLNKDhJmVslBwswq1XE4zzGSviDp3u4hPa9LfU8zG506\nehIfB26MiBOBk+izx6VZ2+3Zs4eNG89h48Zz2LNnT+7mLEvSTWckPQe4MyJ+veIabzpjrbZnzx42\nbz6PgwevAGBycis33LCT6enpzC0rY9OZlwCPSLpW0h2SrpE0mfieZkXZseOaboA4D+gEix07mrMf\ndOqNcFcB64H3R8Ttkj4GXAxs773osssuO/z1hg0bWrtXoFlue/fuZe/evcv6ntTDjTXAv0TECd3n\nbwS2RsTZPdd4uGGt1vThRvKNcCXtA94XEfdL2g48MyK29rzvIGGtt2fPnsNDjJmZLUUECCgnSJwE\nfAo4Gvgu8J6I+FnP+w4SVqxSP9yjUkSQWIqDhJUq5TChlODjIGE2hI0bz2FubhOdWQmAnUxN7ebm\nm7801M8tKUcxSJBIfsyfmR3pyClROHiw81qpQxkHCbMevcOA009fz623buXgwc57k5NbmZnZmbF1\neThImHUtHAbceutWtm27gH37dgMwMzOaIcHMzBZuvfW8xgQf5yRs7CyWNEyVg1hOG+rmnIS10jAf\nsKf3Fs7LkjScnp4uNgfxNBGR9dFpgtlgZmdnY3JyTcB1AdfF5OSamJ2dHfj7p6be3v3e6D6ui6mp\nt4/kZzdR9/NX+Rl1T8IaJeXMwPT0NDfcsLOnl1JG6XRuDhI2VpZKGjZqGFATJy6tUUZRiFRK0rAE\nrri0Vqr6kLcxAKT8nQYJEk5cWmu0MfGY+ndigMSlexLWGnXWOdQl9e9UwvZ1ZtZwnt2w1mhaufMg\nSvidPNywxnHist7EZR07Ux0F3A48GBGb+rzvIGEDK2kvhjYoJSdxIXBPDfexMdD07embKGmQkLQW\nOIvOHpdm1kCpexIfBS4CPJ6wkZiZ2cLk5FZgJ7Czm8jbsuT3NfmYvdySzW5Iegvwo4i4S9IGYNFx\njw/nsUGtZBFWKcvDS1DU4TyS/gL4A+AJYBJ4NvDliHjXguucuLSk2lhkNSpZE5cRcWlEHB+d07vO\nBW5ZGCDMrHwuprLWK6EgqclcTGUr1qTCpSa1tU5FFFMtxUGimVzU1A4OEpaMk4HtUErFpZk1mBOX\ntiJOBo4P9yRsxV75ypeyevXlnHLKtc5HtJh7ErZsC5OWBw9uzdwiS8k9CVu23CsxvQ6jXg4S1ijz\nvZi5uU3MzW1i8+bzDgcKB49EltopN/UD75bdODl3pV7smL7UbZqdnY2pqbcfvldb4GP+LIUSj8NL\nefzfuK8idZCwFcl1HN5iU68pcyIpA1ATOEhYo1T1Yly3kYbLsq01Ui3iavM6Fa/dMBuRtq4idZAw\ns0pe4GVmQ0u+pb6kWyTdLWm/pA+kvJ+ZjV7S4YakFwAviM6O2c8Cvgm8NSK+03ONhxtmmWQfbkTE\nwxFxV/frnwP3AselvKdZPy7ZXrnaEpeSXgzsBV7VDRjzr7snYUm1eQpzWIP0JGoppuoONb4IXNgb\nIOb5cB5LadwrJnsVdTjP4RtIq4CvADdFxMf7vO+ehCXl/TgXV0pP4jPAPf0ChFkdvNXecFLPbpwG\n/DOwn86hwQFcGhGzPde4JzHm6qhmbGvF5LBccWnFc1Ixr+xToDbeBpl2zL0V3ii1dZrVS8UtiXHb\nqKXVv+9SW1elfuDt61ppsW3mFsq5Fd4oDfr7loYBtq/zcMOymt9EZmpqN1NTu5f917etXfyiLBVF\nUj9wT6KV6ughlNQLKakty8EAPQkHCRvYcneMTr3DdGld/CbuqD1IkHDi0gaynMTcuNYk5NocOLml\nokjqB+5JNEKqROQwf32b2sUvCe5JWN2Ws5hq2GnDEs//aCMHCRtIivUPo1id2doufkEcJGwgS/3V\nns9DPPLIj5iYuIhDhzqvezFV83nthg1t4bBhYuJPWbfuJI499nmViUuv28jPC7ysFsPs1zCuMyGl\nKGU/CbNFOadQPpdl29BmZrYwObkV2Ans7OYhtuRu1ki47Lue7evOBD5GJyB9OiKuWPC+hxst0MZh\nwzjkTLLnJCQdBdwPnAH8APg34NzwuRvWAOOwN2YJm86cChyIiO9HxOPA9cBbE9/TzEYodeLyOOCB\nnucP0gkcZsXzBrodnt0wW4TLvjtSB4mHgON7nq/tvnYEH87TPm1JZLZtira4w3kkPQO4j07i8ofA\nN4B3RsS9Pdc4cdkyy50VaEtAaaJBEpd1LAU/k06gOABc3Of9Uax4tYIsZzMYL/fOixL2uIyI2Yh4\nRUS8LCL+MvX9rFnq2lLfRVEr58SljVxpswKt3u6+Dkt1NVI/8HCjlQbdcaqO4UZpe2GWBO9MZbkM\nOivgacbyeam4td44rMFYqexrNwbhIGF18DRrfw4SZlaphAVelpin9iw19yQazGNtG5Z7Eg2ykh7B\nKAuR3COxxThIFGC+RzA3t4m5uU1s3nze4Q9qHR/eqvubuZiqAIsV+yxVaDSqQiQXG40vXEzVbEud\ncOVCJKuDg0QBFlvrMEh+YRT7HczMbGHfvnM5dOiTAExMfIeZmeuH+pnWHg4SBajqEdS3UOpo4I+7\nX1+U6B7WRJ4CLVwdlYLjsCu09ecTvFqgbdunWfMkCxKSPgKcDfwC+E/gPRHxWKr72cqVtv+DlSVl\nncTNwLqIOJnO1nWXJLxXI5RasDSfE5ma2s3U1G5XbdoRaslJSHobcE5E/GGf98YiJ+ESaitRSWXZ\n7wVuquleRaprL0ezURsqSEiak/Ttnsf+7r9n91yzDXg8InYN3dqMSh0qmKU2VOIyIqaq3pf0buAs\n4M1V15V+OM8oNlJ1ctBKUNThPJLOBHYAb4qIn1RcV3xOYlR1BN4dyUqTu07iKmACmJMEcFtEnJ/w\nfsVbTs2DA4qVIlmQiIiXpfrZdat7qOBzIqwkLsseUJ1/2V0mbXXJPdxoFZdH27jyzlQjMOrp0ZmZ\nLUxObgV2Aju7w5stQ/9cs5XwcGNIqSopnbi0OvjcjRr0yx+sXn05u3Zd7Q+2Fa+ksuyx8uijz/dm\nstYa7kkMaeFwA+ZzCQ97RsKK555EDeaXWa9efTnwSToBwsMMaw8HiRGYnp5m166rmZz8HvAwnpGw\nNvFwY4Q8I2FN49kNM6vknISZDc1BoiG86Y3l4uFGA3h/TEvFOYmW8KpQS8U5CTMbWvIgIWlG0lOS\nVqe+10o0YazvVaGWU9LhhqS1wKeAVwC/GRGP9rkm23CjSWN912BYCtlzEpK+APw5sJsCg0Tusb4/\n+JZb1p2pJG0CHoiI/d2NcK2H97G0phgqSEiaA9b0vgQE8CHgUmBqwXt95Tp3I+dZGEee6AUHD3Ze\nc5CwlIo5d0PSq4CvAv9LJzisBR4CTo2IHy+4NusUaK4u//r1b+TOO58EXghsoY6l5R7e2EKDDDeI\niOQP4HvAry3yXoyb2dnZmJh4fsB13cexMTHx3JidnU16z8nJNYfvOTm5Jun9rBm6n7/Kz29du2UH\nFcONcbNjxzUcOnQlv0yYwrp11yb9y+7hja1ULUEiIk6o4z5Nduyxz8vdBLO+fO5GBjkSpj6w2FbK\nazcyyZFEdOLSFspeTDWIcQ0STeLg0l4OEja0JpWu2/J5FegimrCoqxRHzop0gsV8r8LGw9glLl0O\nbbY8Y9eTKOkvYxN6NF6mbmPXkyhFU3o084cP/TJxWV4bLa2xS1yWkojLvUzdDDIvFS+V/zKaLc/Y\n9SRK0a9Hs23bBezbdwfgegSrh+skCtdbpHT66ev58Ievyj4MsvHiINEgzlFYDi6mMrOhjV3islRe\npWml8nCjIF5IZXXLnpOQdAFwPvAE8E8RcXGfaxwkzDLJmpOQtAE4G3h1RLwa+KtU9ypVE8quzZaS\nrCch6e+Av4mIW5a4rpU9iVIqO82q5J7deDnwJkm3SfqapNckvFdxSlpIZjaMlIfzrKKzjf7rJb0W\n+DzQd0PcXIfzmI2bYg7nAZB0I3BFROzrPv8P4HUR8ZMF13m4YZZJ1tkNSVuA4yJiu6SXA3MR8aI+\n17UySICnNK18uYPE0cBngJOBXwAz872KBddlDxL+MNu4yl4nMYjcQcLDAhtnDhID8MIqG2e5p0DN\nrAXGfoGXF1aZVRv74QY4cWnjyzkJM6vknISZDc1BwswqOUiYWSUHCTOr5CBRGG9UY6Xx7EZBXCJu\ndfMUaMO4RNzq5ilQMxva2Jdll8Ql4lYiDzcK4xJxq5NzEmZWKfe5G6+V9A1Jd3b/Havdss3aImXi\n8iPAhyLiFGA7cGXCe5lZIimDxA+BY7pfPxd4KOG9zCyRlBvhHg98nc45HALeEBEP9LnOOQmzTAbJ\nSaQ8nOcC4IKI+HtJ76Czc/ZUv5/jw3nM6lHa4TyPRcRzep7/LCKO6XOdexJmmeSuuDwg6fRuQ84A\n7k94LzNLJGXF5R8BV0uaAP4P2JLwXmaWiIupzMZY7uGGmbWAg4SZVXKQMLNKDhJmVslBwswqOUiY\nWSUHCTOr5CBhZpUcJMyskoOEmVVykDCzSg4SZlbJQcLMKjlImFklBwkzqzRUkJD0Dkn/LulJSesX\nvHeJpAOS7pW0cbhmmlkuw/Yk9gObgX29L0o6Efhd4ETgd4BPSKrc2KJJlruRaAma1uamtRea2eZB\nDBUkIuK+iDhAZ5fsXm8Fro+IJyLiv4ADwKnD3KskTfzP0LQ2N6290Mw2DyJVTuI4oPeMjYe6r5lZ\nwyy5EW7F2RrbIuIfUzXMzMowko1wJX0NmImIO7rPLwYiIq7oPp8FtkfEv/b5Xu+Ca5ZR0hO8Fui9\n0W7gc5I+SmeY8VLgG/2+aakGmllew06Bvk3SA8Drga9IugkgIu4BPg/cA9wInO99882aKfu5G2ZW\ntqIqLiXNSHpK0urcbVmKpI90C8XukvQlSc9Z+rvqJ+lMSd+RdL+krbnbsxRJayXdIuluSfslfSB3\nmwYh6ShJd0janbstg5B0jKQvdP8P3y3pdYtdW0yQkLSWzqnj38/dlgHdDKyLiJPp1IFckrk9TyPp\nKOCvgWlgHfBOSa/M26olPQF8MCLWAb8FvL8BbQa4kM7wuik+DtwYEScCJwH3LnZhMUEC+ChwUe5G\nDCoivhoRT3Wf3gaszdmeRZwKHIiI70fE48D1dArdihURD0fEXd2vf07nP2/RNTbdP3BnAZ/K3ZZB\ndHu9vx0R1wJ0ix4fW+z6IoKEpE3AAxGxP3dbVui9wE25G9HHwqK2Byn8A9dL0ouBk4GnTZ0XZv4P\nXFMSfC8BHpF0bXeIdI2kycUuTnmq+BEqirI+BFxKZ6jR+152gxSSSdoGPB4RuzI0sbUkPQv4InBh\nt0dRJElvAX4UEXdJ2kAh/3eXsApYD7w/Im6X9DHgYmD7YhfXIiKm+r0u6VXAi4FvdReBrQW+KenU\niPhxXe3rZ7E2z5P0bjrdzDfX0qDlewg4vuf52u5rRZO0ik6A+NuI+Ifc7VnCacAmSWcBk8CzJX02\nIt6VuV1VHqTTc7+9+/yLwKJJ7eKmQCV9D1gfET/N3ZYqks4EdgBvioif5G5PP5KeAdwHnAH8kE5B\n2zsjYtEkVQkkfRZ4JCI+mLstyyHpdDqVx5tyt2UpkvYB74uI+yVtB54ZEX0DRW09iWUImtFluwqY\nAOa6q+Bvi4jz8zbpSBHxpKQ/oTMTcxTw6QYEiNOA3wf2S7qTzv+HSyNiNm/LWucDdKqijwa+C7xn\nsQuL60mYWVmKmN0ws3I5SJhZJQcJM6vkIGFmlRwkzKySg4SZVXKQMLNKDhJmVun/AUYHTBJb9HcU\nAAAAAElFTkSuQmCC\n", + "text/plain": [ + "<matplotlib.figure.Figure at 0x7f8040158790>" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "#@test {\"output\": \"ignore\"}\n", + "X += np.random.randn(2, num_examples)\n", + "plt.figure(figsize=(4,4))\n", + "plt.scatter(X[0], X[1])\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "3dc1cl5imNLM" + }, + "source": [ + "## What we want to do\n", + "\n", + "What we're trying to do is calculate the green line below:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "cellView": "form", + "colab": { + "autoexec": { + "startup": false, + "wait_interval": 0 + }, + "output_extras": [ + { + "item_id": 1 + } + ] + }, + "colab_type": "code", + "collapsed": false, + "executionInfo": { + "elapsed": 414, + "status": "ok", + "timestamp": 1446659137254, + "user": { + "color": "#1FA15D", + "displayName": "Michael Piatek", + "isAnonymous": false, + "isMe": true, + "permissionId": "00327059602783983041", + "photoUrl": "//lh6.googleusercontent.com/-wKJwK_OPl34/AAAAAAAAAAI/AAAAAAAAAlk/Rh3u6O2Z7ns/s50-c-k-no/photo.jpg", + "sessionId": "4896c353dcc58d9f", + "userId": "106975671469698476657" + }, + "user_tz": 480 + }, + "id": "P0m-3Mf8sQaA", + "outputId": "74e74f19-6ff8-4a8c-81c7-9021a08b78b5", + "scrolled": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQMAAAEACAYAAAC3RRNlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHvdJREFUeJzt3Xl4VPX1+PH3CRCNRVCgUpWK4iOlRWWxipVa0BDCIsHt\nZ22lLLbybRECGBYXFBBbQeQBRCoWGkWptrUUxQoJQYQWLVKURREBFUGggKDsS0jm/P6YgULMJDOZ\nO3eZnNfzzNNMcrn3tM2cfO7nc+75iKpijDFpXgdgjPEHSwbGGMCSgTEmwpKBMQawZGCMibBkYIwB\n4kgGIvJHEdkpImtO+d65IrJARNaLSKGI1E1OmMaYZItnZPAckF3me/cDC1X1e8Ai4AGnAjPGuEvi\nKToSkcbA66p6ZeT9x0A7Vd0pIt8BFqtqs+SEaoxJpkTnDM5T1Z0AqroDOC/xkIwxXnB6AtFqm40J\nqJoJ/vudItLwlNuEXdEOFBFLFMZ4RFWlsmPiHRlI5HXCXKB35OtewGuVBBSo18iRIz2PIZXjtZjd\necUqnqXFl4B3gKYiskVE+gBjgSwRWQ9kRt4bYwIo5tsEVf15lB91cCgWY4yHrAKxAu3bt/c6hLgE\nLV6wmP0krjqDhC4kom5dyxjzPyKCJmEC0RiToiwZGGMASwbGmAhLBsYYwJKBMSbCkoExBrBkYIyJ\nsGRgjAEsGRhjIiwZGGMASwbGmAhLBsYYwJKBMSbCkoExBnAoGYjIAyKyVkTWiMifRCTdifMaY9yT\ncDKI7KVwD9BKw/sp1ATuTPS8xhh3JdodGWA/UAx8S0RCwFnAdgfOa4xxUcIjA1X9GpgAbAG2AXtV\ndWGi5zXGuCvhkYGINAEGA42BfcDfROTnqvpS2WNHjRp18uv27dunbC85Y7y0ePFiFi9eHPe/S7gH\noojcAWSp6j2R978A2qhq/zLHWQ9EYzzgZg/E9cC1InKmiAjh/RPWOXBeY4yLnJgzWA28ALwHrCa8\n49IfEj2vMcZd1ird+FZhYSETJoT/ruTl9SU7O9vjiIIp1tsESwbGlwoLC7nlll4cOTIOgIyM4cyZ\nM9MSQhVYMjCB1rHjbRQV5RDezxdgJllZc1mwYLaXYQWSbaJijA8UFhbSseNtdOx4G4WFhV6HUyEn\nKhCNcVxeXl+WLu3FkSPh9xkZw8nLm+ltUHEqe6uzdGkvX9/q2G2C8a2gTyD65VYn1tsEGxkY38rO\nzg5cAggySwbGJEnQbnXsNsGYJPLDrY4tLRpjAFtaNMbEyZKBMQawZGCMibBkYIwPFH1axPYD3nYL\ntGRgjIdUlRdXv8ikdydxtOSop7FYMjCBFKSa/2hCGmLisom8vuF18nPyaXJuE0/jsaVFEzip8Hhz\ncWkxoxaP4stDXzIhewJ1zqiTtGu5urQoInVF5BURWRfZTKWNE+c1pjwTJvwhkgh6AeGkcKKwJwgO\nFh8kd34uJaESpnadmtREEA+nypEnA/NU9f+JSE3CeycYY8r48tCX5Bbk0rJhS4a2HUqa+OdO3YlW\n6XWA61W1N4CqlhDeWMWYpAhazf8Jn+/9nNz5udzc7Gb6tOxDuH+wfzjRKr0F4QaoHwEtgBXAQFU9\nUuY4mzMwjvFDzX881uxcw5AFQxhwzQC6fa+bq9d27dkEEbkKWAb8SFVXiMgkYJ+qjixznI4c+b9v\n2SYqprr45+Z/8uiSR3n0hke57rvXJf16ZTdRGT16tGvJoCHwb1VtEnn/Y2C4qnYrc5yNDEy1M2fd\nHKa9N42J2RP5wbd/4EkMrjU3UdWdIvKFiDRV1Q2EN1H5KNHzGhNkqsr096fzxsY3mN5tOhfVvcjr\nkCrl1GpCLvAnEakFfAb0cei8xgROaaiUsUvHsm73Op7r/hz1Mup5HVJMrOjIGAcdLTnKg28+yLGS\nY4zvOJ6zanm/ym79DIxx2b6j+/jNG7+hdnptJnWa5ItEEA/rgWiMA7Yf2M6A+QNo17gd/a/p76ti\nolhZMjAmQRv2bGBQwSB6tujJnZff6XU4VWbJwJgE/Gfbf3hw0YMMu24YWZdmeR1OQiwZGFNFCz5d\nwPh3xjM2cyxXXXCV1+EkzJKBMVXw0gcvMWvNLH7f5fdcVv8yr8NxRPBmOYyJUTIaoIQ0xKRlk5jz\n8Rzyu+enTCIAqzMwKSoZDVCOlx5n9JLRbD+wnUmdJvmmD0FlrM7A+Fqy25Y53QDlUPEhBhYM5GjJ\nUZ7p+kxgEkE8bM7AuC5oW5XvObyHAfMHcPl5l3P/j+8PZA1BTFTVlVf4UsaoZmXdqvC8gkZez2tW\n1q2OXqOgoEAzMhpGrvO8ZmQ01IKCgrjPs3nvZu32Ujed/t50DYVCjsbolshnr9LPqI0MTErKzs5m\nzpyZpzRAiX/k8eGuD7mv8D76Xd2Pm5vdnIwwfcUmEI3rgtDdeOmWpYxaPIqR7UZyfePrvQ4nIbYL\ns/E1P7ctm7t+Lk8vf5oJHSdwRcMrvA4nYZYMjImTqpK/Mp/X1r/GlM5TaHxOY69DcoQtLRoTh5CG\nGLt0LG9uepP87vm+SgRu7R7lWDIQkTQReV9E5jp1TpM6/Lwd2rGSYwwrGsYX+79gerfpNDirgdch\nnXRifqWoKIeiohxuuaVX8v73i2XJIZYXMBiYBcyN8vNkrJqYAHBqmS8Z9h3dp3e/erc+9OZDWlxS\n7HU43+DEMiwxLi06tb1aI6ALMMOJ85nU4pft0MqOTnYc3MEv5/6SKxpewaM3PEqtGrVcj8lPnKoz\nmAgMBeo6dD5jKhXPikTZ5cx/fngXVw5vwqAbB/HzK37uSrxV4ebuUU5sr9YV2Kmqq0SkPRB11nLU\nqFEnv7ZNVKqPsr/QaWmDadcuL6FzxlvSfNro5Pz3ONbhO3w17yg/H+jfRABVK54qu4lKzGK5l6jo\nBfwO2EK4Rfp/gYPAC+UcV9XbJpMCHnvsMU1Lq69wrUJewvMG8d5Lnzy+SZHyiw7KBY9oWlp938xd\nJBNuzRmo6oOqepGGd1S6E1ikqj0TPa9JLUuWvE8oNAH4N/Ckg/MGhcBtwDR2794Z9ai8vL7I5ffC\nj/JgXgfY/iyhUO9AbeWebFZnYHyrouXIvLy+pKcPAnoAOcCvWbt2Q7nLbqrKhnM2UOe6M2FuTdiz\nHJgJBL+60FGxDB+ceGG3CdVavMuLsRzfqlW7Sm8Vjpce14cXPay9X+2tf/vH33y7xJlM2FOLxk/i\nnQg7fTkSjhwJf+/Uf9OgQf0Kr3n4+GGGFQ2jVlotnun6DGfWPJPac2on9CRjSoslYzjxwkYGJg6x\nTBBWNHrYc3iP9vh7Dx2zZIyWlJZ84/wFBQWalXWrZmXdmvKjA2IcGVgyML4U621FQUGBtmrVTuvV\nu1RbtWqrBQUFumXvFu3+cnd9dsWz5TYk8XNFZDJYMjCBF8tf77If7DMa1dfWE1vr7I9mRz2vG52W\n/CTWZGBzBsa3srOzK72nP21u4bvvcOyGhqQtrc2tg251J8gUYsnApIamr0ObKVD4M869cmWFh7pZ\n4hsk1tzEBFpBQQE5I3/K8Usvgfk9yDj2ZEwt1Pzcaclp1unIpLyQhhj/9njmr5oP82qTXnJGyn+w\nq8KSgUlpxaXFjFg0gv3H9vNkxyepnV7b65B8y9qemZS1/9h++s/rTw2pwVOdn7JE4BBLBiZQdh3a\nxa/m/opmDZrx28zfkl4j3euQUoatJpjA+OzrzxgwfwB3Nr+THlf2QKTSka+JgyUDEwgr/7uS4QuH\nM/jawXS+rLPX4aQku00wvlLeY8uLNi1i2MJhjLlhjCWCJLLVBOMb5W27NnjGL3lf32di9kSaNWiW\ntOumcs1BrKsJTjxz0AhYBKwFPgByoxyXhKprk0pOf2YgpFz9Cz1vwAW6dd/WpF2zOjy0hIvPJpQA\n92m4IWpt4D0RWaCqHztwblMdpZXA9b+Fcz/jBxtbc2GdC5N2qVj6JlQXCScDVd0B7Ih8fVBE1gEX\nApYMTFzy8vryr2U9OfrjWaDCmW9u5P5XXvA6rGrD0dUEEbkYaAm86+R5TfVw9U+upu3jV7Dp/W00\n+aIZQ155Iel/oe2hpf9xbAIxcouwGBijqq+V83N16lom9Wzbv43+8/vTsUlHfv3DX7taQ2ATiJHj\nnPiAikhN4B/AfFWdHOUYHTly5Mn3tomKOWHdl+sYXDiYX7X+Fbf/4Havwwm8spuojB492tVk8AKw\nW1Xvq+AYGxmYb1i2dRkjFo3goesf4oZLbvA6nJTk2oNKItIWuAu4UURWRrZl75ToeU3s/LzdeUXm\nbZzHI289wpMdn7RE4AexrD868cLqDBJWXk/AIK6Th0Ihnblqpnb9U1f99KtPvQ4n5WENUVNLtA99\n0Jp7loZKdfzb4/WOV+7QnQd3eh1OtRBrMrAHlQIiWnFMkBSXFvPIW4/w1ZGvmNFtBmefcXbSrpXq\nKwTJYA8qBVxeXl8yMoYT3jtwZmSdvK/XYX3DgWMH6D+vP6rK012eTloiKCwspHXr9nTpchdFRZdQ\nVJTDLbf0CtRcimdiGT448cJuExJS0dyA33cH2nlwp/70lZ/qE0uf0NJQadKuU/Z/I2ioUOD7W6dk\nI8bbBHtqMUCCOPTd9PUmcgtyuf37t9OzRc+kFhN17HgbRUU5nLiVCo+W5gI5ZGXNZcGC2Um7tp/F\nurRocwYBEsumIn6yesdqhhYNZdC1g+hyWRePotherUuM42FzBqZcidYuLP58MUOKhjC6/WjXEkHZ\n+ZO0tMG0alUjpn0UDDZnYL4p0dqF2R/N1uwXs3XtrrVJjLJ8fp8/8QI2Z2Cqqrx771juuVWVZ997\nloJPCni6y9M0qtMo6bGaytmcgXFVaaiU3/3rd2z8aiP53fOpl1HP65BMnCwZmG+I9xn/I8eP8MCb\nD1AaKmXaTdM4q9ZZLkVqnGS3CaZcsS5j7j26l0EFg2hctzEPt3uYmmn298VvbK9Fk3TbD2yn/7z+\nZF6SSb+r+9mmJj5lcwYmqdbvXs+gwkH0admHO5rf4XU4xgGWDEzclm9bzkOLHuL+tveT2STT63CM\nQ6zoKAUls9lJwScFjFg0gnEdxlkiSDFOtT3rBEwinFz+qKrjyjnG5gxcUN6uRE5V4M1aM4uXP3yZ\npzo9xaX1Lk34fMYdrk0gikgasAHIBLYD/wHu1DKbqFgycEdVC4YqEtIQk5ZNYtnWZUzpPIWGtRs6\nEqtxh5sTiNcAG1V1c+TCfwa6Y5uopITi0mJGLR7Fl4e+ZEbODOqcUcfrkEySODFncCHwxSnvt0a+\nZxwSzxyAk81ODhUfYuD8gZSESpjadaolghRnqwk+V3YOYOnSXhXOAWRnZzNnzsxTCoaqNl+w+/Bu\ncufn0qJhC4a2HUqa2FxzqnMiGWwDLjrlfaPI975h1KhRJ7+2TVRiU5WNQU/te3BiVAGxN0TZvHcz\nA+YP4OZmN9OnZR8rJgqYspuoxCyWRxsregE1gE+AxkA6sAr4fjnHOflUZrWRSPfjqjyKvHrHas16\nIUtf+/g1J8I3PoCbrdKBTsB6YCNwf5Rjkv/fOgUl0lsg3kSy5PMlmjkzU5duXupU+MYHYk0GjswZ\nqGoB8D0nzmVO59QcQGVe/fhVnlnxDJM7Tab5ec0dP7/xP3tQKYWUfdIQqLQASVWZ/v503tj4BlM6\nT+GiuhdVeE5rHxY8sdYZWNuzFBHtdqKiNmAlpSX62JLH9K7Zd+mew3tiPqcJFmx7tdQU7cMd7/zA\nkeNHdHDBYO33j356qPhQuccEbes2U75Yk4HVGQRI2ZqDJUt+QfPmTWnQoCG7d++J+Tz7ju5jcOFg\nGtVpxLgO46hVo1ayQjZBEkvGcOKFjQwSVt5farhW4XlNTz9H09O/XemQfvv+7XrrX27VycsmV7q7\nkd0mpAZsZFBdXAD0orgYWrWaToMGc4HyVx027NnAoIJB9GzRkzsvv7PSM7u1kmH8wVYTAqTsbQIM\nAWYB2VT2dOKK7St44M0HGHbdMLIuzXIpYuMH1gMxRZ1Y6tu9ew9r166muHgSUHHfggWfLmD8O+MZ\nmzmWqy64yu2QjccsGVQDsdQAvPTBS8xaM4vJnSZzWf3Lkn494z+WDKq5kIaY8u4U/rXlX0zpPIXz\nzz4/ofM51UHJEor7rOioGisuKdYRb47QPq/20b1H9jpyTidqDmx1whvEuJpgD6kHXNnGJ4ePH2Zg\nwUAOHz/MM12foe6Zdb0O8aTTH8cOjzJOjBKM92xpMcDKDt3/teIX/OjRy+nQogPD2w6nRloNx64V\n75ZrJoBiGT448cJuExx32tC97mblZy31+z1aaSgUSsr1Et3u3G4TvIEVHVUj530IHfNgxfU0+u62\npHUmOrWDUlX/vRUx+ZetJgRYYWEhOf1/RvF158Pi7mR8me/YHgkmdcS6mpDQBKKIPCEi60RklYjM\nFhFrn+uiYxcf4/J7L+HafeeT9b31lghMQhIaGYhIB2CRqoZEZCzhe5MHohxrIwOHqCr5K/N5bf1r\nTOk8hcbnNPY6JONjrmyioqoLT3m7DLgtkfOZyoU0xBNvP8GanWvI755Pg7MaeB2SSRFOTiDeDfzZ\nwfOZMo6VHGPEohEcOn6I6d2m8630b3kdkkkhlSYDESkCTt1cTwAFHlLV1yPHPAQcV9WXKjqX7ZtQ\ndfuP7WdwwWC+U/s7/C7zd9aQxERV1X0TnNh4tTdwD3Cjqh6r4LjAzxl4VVe/4+AOBswfQNvvtiW3\nTa7tbmTi4sqzCYT3S1gL1I/hWEcLKdzmVcHMxj0btfOszjpr9aykX8ttiRYxmdjgRkNUwpumbAbe\nj7x+X8GxLvzXdtapv6ytWrVzvTnoe9vf0w4vdNCCjan3QbFqRPfEmgwSXU1I7AF5Hytb95+Wlufq\n9Rd+tpBxb4/jtzf+lmsuvMbVa7uhKntImuSycuQoyv6yhkIfkJY2mFAo/PNkPqjzlw//wszVM5na\nZSpN6zdNyjWMKcuSQcyuoEWLH1TYcDRRqsrU/0zlrc/fYkbODC44+4Jyj0uFBiH2FKQPxXIv4cSL\ngM0ZuH1Pe7z0uD6y6BHt/Wpv/frI176JK5lsAtEdxDhnYA8qVcCtv8CHjx9meNFwaqbV5PEOj3Nm\nzTOjHtu69Y9ZubKUcIv0vsCOCrsiR5MKowsTG2t7FhB7Du/RHn/voWOWjNGS0pIKjy0oKNC0tHNP\njgqgoUKetR8zFcL2WvS/LXu3aPeXu+uzK56NqSFJeX0I09Lqx/1Btj0Uq5dYk4GVsnnkoy8/4p7X\n76Fni570vaovIvKNfoaxaNHichviG2fEkjGceGEjg5Pe3vK2Zs7M1CWfLzn5vViG7k4N7+02oXrB\nJhD96R8b/sFT7z7Fkx2f5MqGV578fseOt1FUlMOJuoZTt0s7dbKvXbvWLFnyPpDYxJ9NIFYfrvQz\nMLFTVZ5f9Tx///jvPHvTs1xy7iUx/buylZBLl1Zt85KyEu1naFKPJQMXhDTEk+88ycodK8nPyefb\n3/r2N46JVoRjZbvGLTaBmGTFpcXcv/B+Pv3qU6Z3m15uIoD/dQ7OyppLVtZc62doXGdzBkl04NgB\n8hbkUT+jPqNvGE16jfS4z+HUHoem+rKNVz2269AuBswfQJsL2zDo2kEVNiSpbDLPJvtMIiwZeOiz\nrz8jd34uP23+U3pc2aPCTU3sL79JNksGHlm1YxXDioYx+NrBdL6sc6XHV7SkaIwTXNlE5ZSL5YlI\nSETqOXG+oFq0aRFDi4Yy5oYxMSUCY/wk4WQgIo2ALMLtz1JGvKXBr6x9hfHvjGdK5ym0adQm5uvk\n5fUlI2M4MBOYGVlS7Fv1wI2pqljKFCt6Aa8AVwCbgHoVHOdoiWUyxVOuGwqFdOryqXrLn2/Rrfu2\nVvl69ly/SRbcKEcWkRygvareJyKbgKtU9asox2oi13JTrKXBg+77JSsyVvDJV58wudNkzs0417OY\njYnGsXLkCjZRGQE8SPgW4dSfRRX0TVROm/mvWcyiM27npps68vLdL5NRK8Pr8IwBPNhERUQuBxYC\nhwkngUbANuAaVd1VzvGBGRlEW+6bMOEP4RHDmTnQeSB8dZDMdGXhgjkeR2xMdElfTVDVD1X1O6ra\nRFUvAbYCrcpLBEFTYWnw2V9D97th67WwJIc0q+g2KcKxOgMR+Qz4YSrMGUQz49UZ/N+cewmtuAc+\nupr09EE0b96CBg3qW2Wg8S1X6wwAIiOEchNBKnh367v85cBfeLzrGLIu/C+tWk0HarFyZR+KinK4\n5ZZeMXcnckJVuiIZU6FYlhyceBGgpcWy3tjwhma9kKUr/7vy5Pe87CNonYpMPHBje7VUp6q8uOZF\n/rr2r0y7aRpNzm3idUiAbU1mksOSQRQhDTHx3xNZvn05+d3zOe9b5532c9sRyKQae1CpHMWlxYx8\nayR7juxhQscJnH3G2eUe59Wjxfako4mHPbVYRQeLD5JXmMc5Z57DmBvHVKkhiRusx4GJlSWDKth1\naBe583O56vyryLsur8KGJMYEhXVHjtOmrzeRW5DL7d+/nZ4telbYkMSYVGTJAFizcw1DFgxhYJuB\ndG3a1etwjPFEtR8HL/l8CXkL8hjVfhRdm3a1Yh5TfcVSjODECx8WHc3+aLZmv5ita3etVVX/FvNY\nvwOTCGwX5uhCoZBO+8807f5yd92yd8vJ7/txd2K/JigTHLEmg2o3Z1AaKuXxpY+zYc8G8rvnUy/D\n320brdrQuKVaJYOjJUd5YOEDlIRKmHbTNM6qddZpP7eqQlOdVZs6g71H9zK4cDAX1bmIh9s9TM20\n8vOg34p5rNrQJMqKjk6x/cB2+s/rT+YlmfS7ul/gagj8lqBMsFgyiNiwZwODCgbRu2Vv7mh+h+vX\nN8ZrrjU3EZEBIrJORD4QkbGJns9Jy7ct595595L3ozxLBMZUIqEJRBFpD3QDrlDVEhFp4EhUDij8\npJAJ/57AuA7jaH1+a6/DMcb3El1N+A0wVlVLAFR1d+IhJW7Wmlm8/OHLPNP1GS6td6nX4RgTCIne\nJjQFfiIiy0TkLRH5oRNBVdWJhiRz188lPyffEoExcUh0E5WawLmqeq2IXA38FYjaGyyZm6gUlxYz\nevFodh7ayYycGdQ5o45j5zYmSFzfRAVAROYB41R1SeT9J0AbVd1TzrFJW004VHyIIQuGUDu9No/d\n+Bhn1DwjKdcxJojcWk14FbgxcsGmQK3yEkEy7T68m3tev4eLz7mYcVnjLBEYU0WJTiA+B+SLyAfA\nMaBn4iHFbuv+rfR7ox83N7uZPi37BK6YyBg/CXTR0YFjB1i+bTmZTTIdPa+TrHrQeM0qEH2g7HMF\n6elDad68KQ0aNLTEYFzj+vZqbglSJ6LTHz/uRXHxeFauLPVkOzZjKhOoZHDiL21RUU6AP1AXAOHR\nwonbB2P8IFD9DILW6KNsfwQYAszyMCJjogvUyCBosrOzmTNnJllZc2nV6jnS00uAHcDMSOOUvl6H\naMxJgZpADHqjD1tZMF5I2dUE+0AZE5+UTQbGmPik7NKiMSY5LBkYYwBLBsaYCEsGxhjAkoExJsKS\ngTEGsGRgjIlIKBmIyNUislxEVkb+09OGqMaYqkt0ZPAEMEJVWwEjgfGJh+QfVWkq6aWgxQsWs58k\nmgz+C9SNfH0OsC3B8/lK0P5PD1q8YDH7SaKPMN8PvC0iEwi3UL8u8ZCMMV5IdN+EAcAAVX1VRG4H\n8oGsZARqjEmuRPdN2K+qdU55v09V60Y51p5SMsYjsTyolOhtwkYRaaeqS0QkE9iQSDDGGO8kmgz+\nD5gqIunAUcBa9xgTUK71MzDG+JsnFYgikiciIRGp58X1YyUiT4jIOhFZJSKzRcS3u7mKSCcR+VhE\nNojIcK/jqYiINBKRRSKyVkQ+EJFcr2OKlYikicj7IjLX61hiISJ1ReSVyO/xWhFpE+1Y15OBiDQi\nvOKw2e1rV8ECoLmqtgQ2Ag94HE+5RCQNeBrIBpoDPxORZt5GVaES4D5VbQ78CLjX5/GeaiDwkddB\nxGEyME9Vvw+0ANZFO9CLkcFEYKgH142bqi5U1VDk7TKgkZfxVOAaYKOqblbV48Cfge4exxSVqu5Q\n1VWRrw8S/gW90NuoKhf5Q9YFmOF1LLGIjGSvV9XnAFS1RFX3Rzve1WQgIjnAF6r6gZvXdcjdwHyv\ng4jiQuCLU95vJQAfLgARuRhoCbzrbSQxOfGHLCgTbZcAu0XkucitzR9EJCPawY5volJJkdKDnF6U\n5PlyYwXxPqSqr0eOeQg4rqoveRBiyhKR2sDfgIGREYJviUhXYKeqrhKR9vjgdzcGNYHWwL2qukJE\nJhGuGh4Z7WBHqWq5FYgicjlwMbBawnunNwLeE5FrVHWX03HEKlq8J4hIb8JDwxtdCahqtgEXnfK+\nET5/TkREahJOBC+q6mtexxODtkCOiHQBMoCzReQFVe3pcVwV2Up4JL4i8v5vQNTJZc+WFkVkE9Ba\nVb/2JIAYiEgnYALwE1Xd43U80YhIDWA9kEn44bHlwM9UNepkkddE5AVgt6re53Us8RKRdkCequZ4\nHUtlRGQJcI+qbhCRkcBZqlpuQvByr0XF/0OtKUA6UBQezLBMVft5G9I3qWqpiPQnvPqRBvzR54mg\nLXAX8IGIrCT8u/CgqhZ4G1lKygX+JCK1gM+APtEOtKIjYwxgbc+MMRGWDIwxgCUDY0yEJQNjDGDJ\nwBgTYcnAGANYMjDGRFgyMMYA8P8BB4YEUxpBpuwAAAAASUVORK5CYII=\n", + "text/plain": [ + "<matplotlib.figure.Figure at 0x7f80640172d0>" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "#@test {\"output\": \"ignore\"}\n", + "weights = np.polyfit(X[0], X[1], 1)\n", + "plt.figure(figsize=(4,4))\n", + "plt.scatter(X[0], X[1])\n", + "line_x_range = (-3, 5)\n", + "plt.plot(line_x_range, [weights[1] + a * weights[0] for a in line_x_range], \"g\", alpha=0.8)\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "VYUr2uPA9ah8" + }, + "source": [ + "Remember that our simple network looks like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "cellView": "form", + "colab": { + "autoexec": { + "startup": false, + "wait_interval": 0 + }, + "output_extras": [ + { + "item_id": 1 + } + ] + }, + "colab_type": "code", + "collapsed": false, + "executionInfo": { + "elapsed": 170, + "status": "ok", + "timestamp": 1446659140755, + "user": { + "color": "#1FA15D", + "displayName": "Michael Piatek", + "isAnonymous": false, + "isMe": true, + "permissionId": "00327059602783983041", + "photoUrl": "//lh6.googleusercontent.com/-wKJwK_OPl34/AAAAAAAAAAI/AAAAAAAAAlk/Rh3u6O2Z7ns/s50-c-k-no/photo.jpg", + "sessionId": "4896c353dcc58d9f", + "userId": "106975671469698476657" + }, + "user_tz": 480 + }, + "id": "gt8UuSQA9frA", + "outputId": "080025d5-d110-4975-e105-7635afaa3ce9" + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAJYAAABkCAYAAABkW8nwAAAO90lEQVR4Xu2dT5Dc1J3Hv+YQT8VJ\nZUhVdprLWs4FTSrGGv4ql9CuHBCH4GaTFCLZwnIcjOAy8l6Q/1SlU4XHcg6xJgtY2OOik2KxSGoT\nGWrXzYFC2T2MDAtWitRavmQ0e9k2SYGowom4hNRPtqA9TE+rW3/cPfPepcfup6f3fu/Tv9/T+/PV\npo8//vhjsMQsULAFNjGwCrYoKy6xAAOLgVCKBRhYpZiVFcrAYgyUYgEGVilmZYUysBgDpViAgVWK\nWVmhDCzGQCkWGEuwrly5gtf++zW887/vYOn/lnD5T5cT40x9ZQrb/nEbxDtFiHeI2LJlSylGY4X2\nt8BYgUVAvfzqy3i5/TI+vPLhmq37wpYv4AHpATxw3wMMsP4cFJ5jbMAiqA4eOYg/Lv8xMcL26e34\n+vTXk8+vbv1q8n/03TsX38EfLv4h+aRE380dmmNwFY7O2gWOBVgE1Y/2/yjxUls+vwXaY1oS7tZK\n3v94MJ8zceUvV0Dea+H4AoOrQrhGHqxuT0Xjp0P7D2HqH6Yymejyu5dx5PiRZBxGnmt+bj7TdSxT\nfgv0ASuAzglwmyE8pfbZu3VaEDkDdT+AweevzGolvPjvL+LMb84knmr+yHxmqNKyCK7ZQ7OJ5yIo\n+3m6clqx8UrNB1bso2W64FQN9cnijdcdAvNAQWGRPBcLicX3Ua8S84FVcj3PnjuLhRcWkgH63OG5\nXHc7+NTBZEBP47NvffNbucpiF/e3QCaw2g0NfNvES5c+wtQ9u2G0LCj8BLAiFEaeBU0zYJ9fxkfY\njKl7FZgtCzIHIA7QUmXov/g9LmMztt6rwLBMyFROj3TkZ0fgveXh4X96GN//zvf7t2aNHGlI7VlW\n0pYmRC+AKUwAsQu5thOuvIjQEjGBGJ7CQYptdOw6etc6VzXXzcUZwJrGseWt2P28DV2I4OgyDgQK\nFgMTYtQ1xqq10eDuR6j8Fi1NxGTkwpAfRos7h05bQscQIFgibEeHMBHCVhs4EBtY8lQQd6ulvbN7\n8e6f302mC7Z/bXsuo9NkKk1X9PZ+IUyeR0sN4GscYl8DPzOP5VuPYynQwMU+dL4O3wzRbpQQ93O1\nbvQuzgRWS0p/tQA6Nuqcilq7A5u3Px28T7qw7BB1VUHqhEKTB2+pCAIVHZVD3dPgujpE6peOBzes\nQRS5nr/+b//g24nF7JN27qkCGq/J++RknHXm5JlVeiKGr/MQPQMdV0ZkCRBbNUwEMYzQhRyZEHgH\nOv29ynPM6HXtja1Rf7B4AZ7RgZv+SuMAOj+NtrYEX3avfyqMfDi2DdcLEAQBvPOX8MGtR3Ex0MEF\nJiRxP373wWZsvaeBhixDVRrg1/jxlwEWPV3ap+xVrR57Cjgpht2xEDV4mLIFvqkiaoUwwzp4U4Hv\n9/awN7YrR+vuGcAS4ZsdtKV0VNEFVqMLrIkWJGEPPP4hKA0RgiCAc1XsdJQErGQ2Ig7hOQ5sx4Hz\n0u+wvHX2akjtMWCpNhQCiCicq+AcCx1Fh9B2IegcNN6B4Teg1z0EeknzKqPFRe7a9AeLm4ajXvzU\noJEDqUahMESrKxSqbQHbDBGLoXUNlBiuUsNOT8fFQEVsNdHmdOjStTgSGOCnLTQuBDBosLxKqnTw\nntw/glPnoHMS4E6iFVjgbBGcwUGMPAjtawP73GZf/wVkAutYtAvPezYUPoKjipBdGZ5vQOgavGte\nHbfsiXD09TZUIUbg6JD3vITlrU/iYthErPOYaQk44ZhocDF8U0HDqsEOHfQaC7/2X68lyzJVTjd0\nWiJu2XMem++7+tAxSd52+hguTe3GYtjq6V3XPyqDtbA/WLyAtqRg0rHhLceo3avCsk0kjqd7uoEL\n0FJkaC/9Hh/gS9ixS0dTCaDKHVidNhoTNN2gQP/FedAmly/t2IWm2YK2xswqDbj3antzz5oToD/9\n15/i5smbcdo8vfaDQGiC37YfEyeW4KtcMu2g1HbCrp9Dx5Fw3ZCw04ZSb0Jse6CsLH1qgZFfK0zn\nn+hpznzKHGpJRzus4YJ/AX/78G94ofUC7r777pwMxAhdE6pyAK8u78CJJZ+BtcKiIw8Wea0DTx34\nZCH5oHYwM1y0TjhnziXbaWgB+4cP/RCPPfYYtm/fjpMnT+Kmm24aDrDYhdpoQdAbaMtNSB4Da6Uh\nRx4sqnB3SCTPNbtvtu9iMoU/Wg5Kt9p0h8DTp09j3759ePrpp/H4448PB1fylOtC5jTUGVifseFY\ngJXClXou+jcN6Gk2nj7JG1Gi7TG0Hkiz7OlGP/ru6OGjq46rnnjiCSwuLibe66677hocMAZWT5uN\nDVgpXGfbZ5OtybQNZq1EE6G0NXmXtGvNwbrv+4n3uu222wYPjwys9QFW2goKjbQ4Tdth6CAFeSpK\n5J3oQMUwhynS8PjMM89AVdVs3ouBtb7Aytbrw+WiMZfnednCIwOLgTUIZml43LFjB5577rnhnx4H\nuek6yztWY6yqbb+wsJBMTwwUHquu5Ijej4GVoWMoPJ4/fz7xXkM9PWa4x3rLwsDK2KMXLlxIvBeF\nR5qe2LRpU8YrN2Y2BtaA/U7hkaYnnn322exPjwPeYz1kZ2AN2YtpeCTvdeeddw5Zyvq9jIGVo28p\nPJL3ok2NLDxeb0gGVg6w0kvT8HjixIlkHJY1lauaE8GRangwsvD/noKqt+kzsLJSkCEfzdi/8cYb\nifdaKzxWoppDmxJ5FT54NH06YZShAQVmYWAVaEwqKg2PMzMzyfTEyqfHqlRzAoOH6OqwJnXoNQeB\nSWcjq0sMrJJsferUqSQsdofHylRzYg8aLyG0QtiTOvhGhFZglyKD0Mt8DKySwEqLpfD45ptvYn5+\nHr/+z19/sukwj2pOP72vyJXBy4BNME340Pg6AiNAu8IDkQysksGi4t9++2189wffxee++DkIO4Tc\nqjlrSw504Eg81FobYetq+KOwKDgagjVOnRdtBgZW0RZdpbw0BL73/nv4yZM/6bv7tVeVxkk1h4FV\nAVgbUTWHgVUBWGUcvCVV6EP/cuiztQ9NCNsMiIshrPSIeaK3oUNIlXQqaDMDqwIjlyEV0Fv6MoQl\nbENT/FTIhWSXOF2AF5jocei8cCswsAo36WcLLEPchO7yyr+9smrt6TQ3geQmcgcd2CQbIHoIDKGy\nuSwG1joEi06oU+jj3RAWR2HQgFiiTuxqJmRgVQBWGaGQDo78/OjPe9T+qpfSeBeeqIM3JPip4k8F\n7aVbMLAqMHSlg/dr7YkcCZxWg1Jz0G5UL7/EwKoArBuhmoNEbupBvPrRDhxf8qFVLFrCwKoArFQi\n4P3o/VwTpCmgdBi3r2oOIrQbNdwfGljytZ46r2U1n4FVlmW7yn3rrbfwvX/+XrKkMyPM5FLNIS2K\nbCrSNI8loKX48G6AxhIDq2SwaIcDgWWaJn71H78qRDWnlxbF1aaQxJILj6TRjRhm0L4hYrwMrJLA\nos1+BBXtyaLty5SKVs1Zverx1RB4dhIPPe/CVioeXF2rFAOrYLDIOxFQd9xxRwLVytSt90XfFaGa\nU3ATCimOgVWIGa8WkoY9AorA6pUIrqJVcwpsRiFFMbAKMONqYS9LsWWo5mS5bxV5GFg5rExhj8ZP\ndHBitbCXo+ixv5SBNWQXpmGPvNXtt98+ZCnr9zIG1oB9O2zYG/A2Y5+dgZWxC1nYy2goNt2Q3VA0\njqIDESzsZbcZ81hr2CoNe/T56KOPZrcqy8m2zazGAAt7+X8ZzGOtsCELe/mhohLGEqwyVFpY2CsG\nqLSUsQKrDJUWFvaKBWrswCpDpYWFvXKgKiYUxh5U/huwhd8idBqYRARX4bHTldd8Le8gTSpapYWW\nX0is47qnveTdi02I6aFOejlAbSdcOT2fF8NTOEixDTqnV6Uk0CC2GpW8hYTCyFXA72yj8XoAAzoE\n+nsxgNnrZc8DtL7bU9HJlDwqLY9855FkbY8ktS3LWlGLECbPo6UG8DUOsa+Bn5nH8q3HsRRo4GIS\nL6vDN0O0e70SdoB2rfeshYBF71Juyzzu90TcF59FIC8WJvSVvgiT9nnPH5nP/K7CtOPonYWzh2aT\nF2Fu+usmvPjLF3us7cXwdR6iZ6DjyogsAWKrhokghhG6kCMTAu9Ap7+r1l0cQwoLAote4+ugwT+I\nsxO78XrQKkTkqzsEkqeily8Nk0il5cfHfowv3/xlLBxf6Pk2sNhTwEkx7I6FqMHDlC3wTRVRK4QZ\n1sGbCnxfrfxgwjBtvtHXFAZW7OsQZo7hEm7Fkxf8nm+mH6TBlau0RG00OBWcY6Gj6BDaLgSdDn46\nMPwG9Hr15/MGsdco5S0GrDiAIU7D5M/AgIo9gY6Lng4+5wi3jIOea59wieCQzgEnAe4kWoEFzhbB\nGRzEyIPQDmBWpaoxSpQMUZdCwCLh1OlmDWcCBzJsSNzDiIyL8LR8Ur1lHE2nPeZzh+d6mooENW7Z\ncx6b7zuHTlvCJB1Nnz6GS1O7sUhKxDl/LEP00Vhekh8sUjThNUyYAdxr59dCSwSvAWbg5Xq7exkq\nLfRO6TMnz/TurNAEv20/Jk4swaf2xC6U2k7Y9XPoOBIm6crYh6UoaLodABOoSU3YlpLbQ48lQT0q\nnR+sEq1RBlj0dGmfsnPVOtB51IMmfEdGLQ7RkkSYkps8VbJ01QIjDdaNCIVZwOi4DnxOgsRRXIzh\nazwakY3gmphsljLWe56RBqv6wfvg3R0HFqS6CcHxC5kQHrwGo3nFSIN1Q1RaBuinyDchSyYmDRct\nhWPLPF22G2mwuo+k55kgHUylJRtZoa1A0kI0bAdGPRnSszQuYFE90yUdepoznzKHWtLRDmsglZY8\ncHZTE7UVCGqEpmtDScZZLK20wEh7LKpst9YBKQUf1A5mhovWCefMuU9eM9JbWnEQMAIY/DQOXLr+\nmqmHXkfIdj18YpSRByuFa6+2F1f+cgXkuWb3zfZdN6Twt/DCQuKpsgmVDQIXy9vPAmMB1krPRf9e\nryot/TpsXL4fG7BSuNa7Ssu4gNOvnmMFVtqY9azS0q/DxuX7sQRrXIy7kevJwNrIvV9i2xlYJRp3\nIxfNwNrIvV9i2xlYJRp3IxfNwNrIvV9i2xlYJRp3IxfNwNrIvV9i2xlYJRp3Ixf9d0NIelzdt4X5\nAAAAAElFTkSuQmCC\n", + "text/plain": [ + "<IPython.core.display.Image object>" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from IPython.display import Image\n", + "import base64\n", + "Image(data=base64.decodestring(\"iVBORw0KGgoAAAANSUhEUgAAAJYAAABkCAYAAABkW8nwAAAO90lEQVR4Xu2dT5Dc1J3Hv+YQT8VJZUhVdprLWs4FTSrGGv4ql9CuHBCH4GaTFCLZwnIcjOAy8l6Q/1SlU4XHcg6xJgtY2OOik2KxSGoTGWrXzYFC2T2MDAtWitRavmQ0e9k2SYGowom4hNRPtqA9TE+rW3/cPfPepcfup6f3fu/Tv9/T+/PVpo8//vhjsMQsULAFNjGwCrYoKy6xAAOLgVCKBRhYpZiVFcrAYgyUYgEGVilmZYUysBgDpViAgVWKWVmhDCzGQCkWGEuwrly5gtf++zW887/vYOn/lnD5T5cT40x9ZQrb/nEbxDtFiHeI2LJlSylGY4X2t8BYgUVAvfzqy3i5/TI+vPLhmq37wpYv4AHpATxw3wMMsP4cFJ5jbMAiqA4eOYg/Lv8xMcL26e34+vTXk8+vbv1q8n/03TsX38EfLv4h+aRE380dmmNwFY7O2gWOBVgE1Y/2/yjxUls+vwXaY1oS7tZK3v94MJ8zceUvV0Dea+H4AoOrQrhGHqxuT0Xjp0P7D2HqH6Yymejyu5dx5PiRZBxGnmt+bj7TdSxTfgv0ASuAzglwmyE8pfbZu3VaEDkDdT+AweevzGolvPjvL+LMb84knmr+yHxmqNKyCK7ZQ7OJ5yIo+3m6clqx8UrNB1bso2W64FQN9cnijdcdAvNAQWGRPBcLicX3Ua8S84FVcj3PnjuLhRcWkgH63OG5XHc7+NTBZEBP47NvffNbucpiF/e3QCaw2g0NfNvES5c+wtQ9u2G0LCj8BLAiFEaeBU0zYJ9fxkfYjKl7FZgtCzIHIA7QUmXov/g9LmMztt6rwLBMyFROj3TkZ0fgveXh4X96GN//zvf7t2aNHGlI7VlW0pYmRC+AKUwAsQu5thOuvIjQEjGBGJ7CQYptdOw6etc6VzXXzcUZwJrGseWt2P28DV2I4OgyDgQKFgMTYtQ1xqq10eDuR6j8Fi1NxGTkwpAfRos7h05bQscQIFgibEeHMBHCVhs4EBtY8lQQd6ulvbN78e6f302mC7Z/bXsuo9NkKk1X9PZ+IUyeR0sN4GscYl8DPzOP5VuPYynQwMU+dL4O3wzRbpQQ93O1bvQuzgRWS0p/tQA6Nuqcilq7A5u3Px28T7qw7BB1VUHqhEKTB2+pCAIVHZVD3dPgujpE6peOBzesQRS5nr/+b//g24nF7JN27qkCGq/J++RknHXm5JlVeiKGr/MQPQMdV0ZkCRBbNUwEMYzQhRyZEHgHOv29ynPM6HXtja1Rf7B4AZ7RgZv+SuMAOj+NtrYEX3avfyqMfDi2DdcLEAQBvPOX8MGtR3Ex0MEFJiRxP373wWZsvaeBhixDVRrg1/jxlwEWPV3ap+xVrR57Cjgpht2xEDV4mLIFvqkiaoUwwzp4U4Hv9/awN7YrR+vuGcAS4ZsdtKV0VNEFVqMLrIkWJGEPPP4hKA0RgiCAc1XsdJQErGQ2Ig7hOQ5sx4Hz0u+wvHX2akjtMWCpNhQCiCicq+AcCx1Fh9B2IegcNN6B4Teg1z0EeknzKqPFRe7a9AeLm4ajXvzUoJEDqUahMESrKxSqbQHbDBGLoXUNlBiuUsNOT8fFQEVsNdHmdOjStTgSGOCnLTQuBDBosLxKqnTwntw/glPnoHMS4E6iFVjgbBGcwUGMPAjtawP73GZf/wVkAutYtAvPezYUPoKjipBdGZ5vQOgavGteHbfsiXD09TZUIUbg6JD3vITlrU/iYthErPOYaQk44ZhocDF8U0HDqsEOHfQaC7/2X68lyzJVTjd0WiJu2XMem++7+tAxSd52+hguTe3GYtjq6V3XPyqDtbA/WLyAtqRg0rHhLceo3avCsk0kjqd7uoEL0FJkaC/9Hh/gS9ixS0dTCaDKHVidNhoTNN2gQP/FedAmly/t2IWm2YK2xswqDbj3antzz5oToD/915/i5smbcdo8vfaDQGiC37YfEyeW4KtcMu2g1HbCrp9Dx5Fw3ZCw04ZSb0Jse6CsLH1qgZFfK0znn+hpznzKHGpJRzus4YJ/AX/78G94ofUC7r777pwMxAhdE6pyAK8u78CJJZ+BtcKiIw8Wea0DTx34ZCH5oHYwM1y0TjhnziXbaWgB+4cP/RCPPfYYtm/fjpMnT+Kmm24aDrDYhdpoQdAbaMtNSB4Da6UhRx4sqnB3SCTPNbtvtu9iMoU/Wg5Kt9p0h8DTp09j3759ePrpp/H4448PB1fylOtC5jTUGVifseFYgJXClXou+jcN6Gk2nj7JG1Gi7TG0Hkiz7OlGP/ru6OGjq46rnnjiCSwuLibe66677hocMAZWT5uNDVgpXGfbZ5OtybQNZq1EE6G0NXmXtGvNwbrv+4n3uu222wYPjwys9QFW2goKjbQ4Tdth6CAFeSpK5J3oQMUwhynS8PjMM89AVdVs3ouBtb7Aytbrw+WiMZfnednCIwOLgTUIZml43LFjB5577rnhnx4Huek6yztWY6yqbb+wsJBMTwwUHquu5Ijej4GVoWMoPJ4/fz7xXkM9PWa4x3rLwsDK2KMXLlxIvBeFR5qe2LRpU8YrN2Y2BtaA/U7hkaYnnn322exPjwPeYz1kZ2AN2YtpeCTvdeeddw5Zyvq9jIGVo28pPJL3ok2NLDxeb0gGVg6w0kvT8HjixIlkHJY1lauaE8GRangwsvD/noKqt+kzsLJSkCEfzdi/8cYbifdaKzxWoppDmxJ5FT54NH06YZShAQVmYWAVaEwqKg2PMzMzyfTEyqfHqlRzAoOH6OqwJnXoNQeBSWcjq0sMrJJsferUqSQsdofHylRzYg8aLyG0QtiTOvhGhFZglyKD0Mt8DKySwEqLpfD45ptvYn5+Hr/+z19/sukwj2pOP72vyJXBy4BNME340Pg6AiNAu8IDkQysksGi4t9++2189wffxee++DkIO4TcqjlrSw504Eg81FobYetq+KOwKDgagjVOnRdtBgZW0RZdpbw0BL73/nv4yZM/6bv7tVeVxkk1h4FVAVgbUTWHgVUBWGUcvCVV6EP/cuiztQ9NCNsMiIshrPSIeaK3oUNIlXQqaDMDqwIjlyEV0Fv6MoQlbENT/FTIhWSXOF2AF5jocei8cCswsAo36WcLLEPchO7yyr+9smrt6TQ3geQmcgcd2CQbIHoIDKGyuSwG1joEi06oU+jj3RAWR2HQgFiiTuxqJmRgVQBWGaGQDo78/OjPe9T+qpfSeBeeqIM3JPip4k8F7aVbMLAqMHSlg/dr7YkcCZxWg1Jz0G5UL7/EwKoArBuhmoNEbupBvPrRDhxf8qFVLFrCwKoArFQi4P3o/VwTpCmgdBi3r2oOIrQbNdwfGljytZ46r2U1n4FVlmW7yn3rrbfwvX/+XrKkMyPM5FLNIS2KbCrSNI8loKX48G6AxhIDq2SwaIcDgWWaJn71H78qRDWnlxbF1aaQxJILj6TRjRhm0L4hYrwMrJLAos1+BBXtyaLty5SKVs1Zverx1RB4dhIPPe/CVioeXF2rFAOrYLDIOxFQd9xxRwLVytSt90XfFaGaU3ATCimOgVWIGa8WkoY9AorA6pUIrqJVcwpsRiFFMbAKMONqYS9LsWWo5mS5bxV5GFg5rExhj8ZPdHBitbCXo+ixv5SBNWQXpmGPvNXtt98+ZCnr9zIG1oB9O2zYG/A2Y5+dgZWxC1nYy2goNt2Q3VA0jqIDESzsZbcZ81hr2CoNe/T56KOPZrcqy8m2zazGAAt7+X8ZzGOtsCELe/mhohLGEqwyVFpY2CsGqLSUsQKrDJUWFvaKBWrswCpDpYWFvXKgKiYUxh5U/huwhd8idBqYRARX4bHTldd8Le8gTSpapYWWX0is47qnveTdi02I6aFOejlAbSdcOT2fF8NTOEixDTqnV6Uk0CC2GpW8hYTCyFXA72yj8XoAAzoE+nsxgNnrZc8DtL7bU9HJlDwqLY9855FkbY8ktS3LWlGLECbPo6UG8DUOsa+Bn5nH8q3HsRRo4GISL6vDN0O0e70SdoB2rfeshYBF71Juyzzu90TcF59FIC8WJvSVvgiT9nnPH5nP/K7CtOPonYWzh2aTF2Fu+usmvPjLF3us7cXwdR6iZ6DjyogsAWKrhokghhG6kCMTAu9Ap7+r1l0cQwoLAote4+ugwT+IsxO78XrQKkTkqzsEkqeily8Nk0il5cfHfowv3/xlLBxf6Pk2sNhTwEkx7I6FqMHDlC3wTRVRK4QZ1sGbCnxfrfxgwjBtvtHXFAZW7OsQZo7hEm7Fkxf8nm+mH6TBlau0RG00OBWcY6Gj6BDaLgSdDn46MPwG9Hr15/MGsdco5S0GrDiAIU7D5M/AgIo9gY6Lng4+5wi3jIOea59wieCQzgEnAe4kWoEFzhbBGRzEyIPQDmBWpaoxSpQMUZdCwCLh1OlmDWcCBzJsSNzDiIyL8LR8Ur1lHE2nPeZzh+d6mooENW7Zcx6b7zuHTlvCJB1Nnz6GS1O7sUhKxDl/LEP00Vhekh8sUjThNUyYAdxr59dCSwSvAWbg5Xq7exkqLfRO6TMnz/TurNAEv20/Jk4swaf2xC6U2k7Y9XPoOBIm6crYh6UoaLodABOoSU3YlpLbQ48lQT0qnR+sEq1RBlj0dGmfsnPVOtB51IMmfEdGLQ7RkkSYkps8VbJ01QIjDdaNCIVZwOi4DnxOgsRRXIzhazwakY3gmphsljLWe56RBqv6wfvg3R0HFqS6CcHxC5kQHrwGo3nFSIN1Q1RaBuinyDchSyYmDRcthWPLPF22G2mwuo+k55kgHUylJRtZoa1A0kI0bAdGPRnSszQuYFE90yUdepoznzKHWtLRDmsglZY8cHZTE7UVCGqEpmtDScZZLK20wEh7LKpst9YBKQUf1A5mhovWCefMuU9eM9JbWnEQMAIY/DQOXLr+mqmHXkfIdj18YpSRByuFa6+2F1f+cgXkuWb3zfZdN6Twt/DCQuKpsgmVDQIXy9vPAmMB1krPRf9eryot/TpsXL4fG7BSuNa7Ssu4gNOvnmMFVtqY9azS0q/DxuX7sQRrXIy7kevJwNrIvV9i2xlYJRp3IxfNwNrIvV9i2xlYJRp3IxfNwNrIvV9i2xlYJRp3IxfNwNrIvV9i2xlYJRp3Ixf9d0NIelzdt4X5AAAAAElFTkSuQmCC\"), embed=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "Ft95NDUZy4Rr" + }, + "source": [ + "That's equivalent to the function $\\hat{y} = w_2 x + w_1$. What we're trying to do is find the \"best\" weights $w_1$ and $w_2$. That will give us that green regression line above.\n", + "\n", + "What are the best weights? They're the weights that minimize the difference between our estimate $\\hat{y}$ and the actual y. Specifically, we want to minimize the sum of the squared errors, so minimize $\\sum{(\\hat{y} - y)^2}$, which is known as the *L2 loss*. So, the best weights are the weights that minimize the L2 loss." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "RHDGz_14vGNg" + }, + "source": [ + "## Gradient descent\n", + "\n", + "What gradient descent does is start with random weights for $\\hat{y} = w_2 x + w_1$ and gradually moves those weights toward better values.\n", + "\n", + "It does that by following the downward slope of the error curves. Imagine that the possible errors we could get with different weights as a landscape. From whatever weights we have, moving in some directions will increase the error, like going uphill, and some directions will decrease the error, like going downhill. We want to roll downhill, always moving the weights toward lower error.\n", + "\n", + "How does gradient descent know which way is downhill? It follows the partial derivatives of the L2 loss. The partial derivative is like a velocity, saying which way the error will change if we change the weight. We want to move in the direction of lower error. The partial derivative points the way.\n", + "\n", + "So, what gradient descent does is start with random weights and gradually walk those weights toward lower error, using the partial derivatives to know which direction to go." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "W7SgnPAWBX2M" + }, + "source": [ + "## The code again\n", + "\n", + "Let's go back to the code now, walking through it with many more comments in the code this time:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "cellView": "both", + "colab": { + "autoexec": { + "startup": false, + "wait_interval": 0 + }, + "output_extras": [ + { + "item_id": 1 + } + ] + }, + "colab_type": "code", + "collapsed": false, + "executionInfo": { + "elapsed": 718, + "status": "ok", + "timestamp": 1446659172854, + "user": { + "color": "#1FA15D", + "displayName": "Michael Piatek", + "isAnonymous": false, + "isMe": true, + "permissionId": "00327059602783983041", + "photoUrl": "//lh6.googleusercontent.com/-wKJwK_OPl34/AAAAAAAAAAI/AAAAAAAAAlk/Rh3u6O2Z7ns/s50-c-k-no/photo.jpg", + "sessionId": "4896c353dcc58d9f", + "userId": "106975671469698476657" + }, + "user_tz": 480 + }, + "id": "4qtXAPGmBWUW", + "outputId": "0664707f-ea8a-453b-fc3f-48d5ca0f76dc" + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlUAAAEPCAYAAABr+zG+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl8VPW9//HXJwtLIKwBQVAEEdlETBUXLM2VKlZR22uv\n2NpatNdb6lqtGm17L1j7uwgu1KU+qPe617Vqi3LdEIm4VFHjggNaXNjXsIU1ZPn8/pgJjiEJSWY5\nk5n38/GYR2bOnDnnM0K+fviez/l8zd0RERERkdhkBR2AiIiISDpQUiUiIiISB0qqREREROJASZWI\niIhIHCipEhEREYkDJVUiIiIicRCXpMrMrjezkJl9bGaPmFmbeBxXRCRRzKyvmb0aGbsWmtnlke1d\nzexlM/vMzF4ys85Rn7nezJaY2WIzOyW46EUkFcWcVJlZP+Ai4Ch3HwHkAOfGelwRkQSrAq5y92HA\n8cAlZjYYuA54xd0PB14Frgcws6HAOcAQ4HvA3WZmgUQuIikpHjNV5cAeoIOZ5QB5wOo4HFdEJGHc\nfa27fxh5vh1YDPQFzgIejOz2IPD9yPMzgcfdvcrdlwJLgFFJDVpEUlrMSZW7bwZuBZYDq4At7v5K\nrMcVEUkWMzsEGAm8DRzg7usgnHgBPSO79QFWRH1sVWSbiAgQn8t/A4ArgX7AgUBHM/txrMcVEUkG\nM+sIPAVcEZmxqrt2l9byEpEmyYnDMY4G3nT3TQBm9gxwAvBo9E5mpoFJJE24e1rUEkVKFp4CHnb3\nWZHN68zsAHdfZ2a9gPWR7auAg6I+3jeyre4xNdaJpInmjnXxqKn6DDjOzNpFijbHEq5NqC+4lHhM\nnjw58BhSKQ7FktpxpFosaeY+YJG73x617VlgYuT5z4BZUdvPNbM2ZtYfGAgsqO+gQf8ZpdPft0yM\nPx2+Q2uP371lY13MM1Xu/pGZPQS8D1QDHwD3xHpcEZFEMrPRwHnAQjP7gPBlvt8A04AnzexCYBnh\nO/5w90Vm9iSwCKgELvaWjrwikpbicfkPd78ZuDkexxIRSQZ3fxPIbuDt7zbwmanA1IQFJSKtWkZ2\nVC8qKgo6BCB14gDFUp9UiQNSKxZJf63971trjx9a/3do7fG3lCVr9trMNFMukgbMDE+TQvVE0Fgn\nkh5aMtZl5EyViIiISLwpqRIRERGJAyVVIimuoqqixbf3iohI8iipEklhVTVV3LXgLhasqrcdkoiI\npBAlVSIpyt154MMH6NCmA8f0OSbocEREZD+UVImkqGcWP8OmXZv4+VE/J8v0qyoikuo0UoukoHlf\nzeOjdR9xyTGXkJudG3Q4IiLSBEqqRFLMB2s+4MXPX+TyYy+nQ5sOQYcjIiJNpKRKJIV8sekL/vLx\nX7j4mIspyCsIOhwREWkGJVUiKWLd9nXMfG8mFxx1Af269As6HBERaSYlVSIpoLyinDveuYPvD/4+\nw3sODzocERFpASVVIgGrqKrgrgV3cVzf4xh98OigwxERkRZSUiUSoBqv4Z7376Fvp76MHzQ+6HBE\nRCQGSqpEAuLuPPLxIzjOeUech1mzFkMXEZEUo6RKJCDPL3me5VuX84tv/YLsrOygwxERkRgpqRIJ\nwFsr3uLNFW9y2bGX0TanbdDhiIhIHCipEkmy0PoQzyx+hsuPvZxObTsFHY6IiMRJXJIqM+tsZn81\ns8VmFjKzY+NxXJF0s3zrcu774D4mHT2JXh17BR1ORjOze81snZl9HLVthJm9ZWYfmdksM+sY9d71\nZrYkMs6d0tix3RMZuYikKvM4/Pab2QPAa+5+v5nlAHnuXl5nH4/HuURKS0t54omXAJgwYRyFhYUB\nR9Q0G3duZPqb05kwfAKFvVtHzPUxM9y91VfVm9mJwHbgIXcfEdm2ALjK3d8ws4nAAHf/LzMbCjwC\nHAP0BV4BDqtvUDMz377d6aAVhkRatZaMdTHPVJlZJ+Db7n4/gLtX1U2oROKltLSUSZNmMH/+KObP\nH8WkSTMoLS0NOqz92rFnB3e8cwfjBo5r1QlVOnH3N4DNdTYfFtkO4cTp7MjzM4HHI+PbUmAJMKqh\nY2/bFudgRaRViMflv/5AmZndb2alZnaPmbWPw3FF9vHEEy+RnT2RgoKxFBSMJTt74t5Zq1RVWV3J\n3e/ezREHHMFJ/U8KOhxpXMjMzow8P4fwrBRAH2BF1H6rItvqpaRKJDPlxOkYhcAl7v6emf0RuA6Y\nHIdji7Rq7s59H9xHl3ZdOHvI2fv/gATtQuBOM/tP4FlgT0sOcsstU+jdO/y8qKiIoqKieMUnIglS\nUlJCSUlJTMeIR1K1Eljh7u9FXj8FFNe345QpU/Y+10AjLTFhwjjmzZtBWVn4dXX1A0yYcGWwQdUR\nCoWYPWc2ALlDcqnJr+GKY69otc094zHQtBbu/k9gHICZHQacHnlrFXBQ1K59I9vq9eMfT+E730lU\nlCKSCHXzkhtuuKHZx4hXofprwEXu/k8zm0y4UL24zj4qVJe4eOSRR7jjjicAuPzyCZx33nlxPX4s\nhfChUIhrb76W3GG5rK9ez4a1G3jsgsc4+sij4xpjkNKlUB3AzA4BnnP3IyKve7j7BjPLAu4H5rn7\nA1GF6scSvuw3h0YK1Z97zhmvVYdEWrWWjHXxmKkCuBx4xMxygS+BC+J0XJFvKC0t5fbbXyQ7+woA\nbr/9AYYMGbI38Yn1zsDaQvjs7IkAzJs3g5kzr2zycWbPmU3usFyyDspi55adHJZ1GHPnzU2rpCpd\nmNmjQBHQ3cyWEy5ZyDezSwAHnnH3BwDcfZGZPQksAiqBixv7V6JqqkQyU1ySKnf/iPCtxiIJFV2o\nDlBWFt5WWFgYc0K0v+M31faa7WzYvIHhPYezY9uOJn9Oksvdf9zAW3c0sP9UYGpTjq2kSiQzqaO6\npI1UuDPw2BOP5ct1X1JQXsCOL3dQGapk/Mm6DpRplFSJZKZ4Xf4TSYpEF6rHcvwtu7fw8taX+cP3\n/8CK0vDd9+OvGc+wYcPiFp+0DkqqRDJTXArVm3QiFapLnDRUN1X38l919QPNvvzX2PEbs7tqN7e8\ndQuFvQs57bDTmnW+1iadCtUTwcz8qqucW28NOhIRiUVLxjolVZJWgljCprqmmrsW3EX3vO6cd8R5\nrbZ1QlMpqWqcmflFFzn33BN0JCISiyDv/hNJCYWFhQlLpOpL2Nydhz9+mOysbH58xI8xs1a7NqHE\njy7/iWQmJVUiTdDQnYUrO6xkzbY1XHX8VWRZVlzuQJTWr1yrn4pkJCVVIk1QX6uFW/46kwEn96R4\ndDFtc9o2uF9zWzJI66eZKpHMpJYKIi2ws+OXfJ79KZcfezn5bfODDkdSjJIqkcykmSqRJpgwYRzP\nPz+ZVRsfp6rDNvYMfZf7xs2gZ4ee++yX6msTSuIpqRLJTEqqROoRXWx+9NED+cd7/2Bb3kJqRi5k\nV8E2Dll7MId2O3SfzxUWFjJz5pVRheqqp8pESqpEMpNaKojUEV1svnPnUpbt/C+6DWzLjn472N1t\nN0f1O4ou5V04Pvd4in9VvN/jpRu1VGicmXn79s7OnUFHIiKxaMlYp5oqkTqii833ZJWRNfgQdloF\ne7ruoW37tuzatCvoECXFVVRAVVXQUYhIsimpEtkPx/HDHVtlZK3MYtvybVrTTxrVsSNs3x50FCKS\nbEqqROqYMGEcZWU38nrpKFasepid2z8mv2MeI3qMoMsnXTi1y6lMv2a61vSTBuXnq65KJBOpUF2a\nJRO6hb/55pus3PMO1X2q4VDDKmDUulEcOfRIxl+oBZJl/5RUiWQmJVXSZJnQLTwUCvH7//k9NSfW\nkNM9h6rtVXSo6cjmTZszsihdWkZJlUhmUlIlTZYJ3cJnz5mNHW5YX6Mmv4bcmlwqP6+Envv/rEgt\nJVUimUk1VSJ19OnXh6rKKtgGVZurqFpWRfuaAoqLp1JaWhp0eNIKKKkSyUxKqqTJJkwYR3X1A5SV\nzaWsbG6kW/i4oMOKq29/59uUV5dzePbhdPqyE3kf5NHDh7NmzUTmzx/FpEkzlFjJfimpEslMcbv8\nZ2ZZwHvASnc/M17HldSRrt3CQ6EQs+fMptIrWdt3LcXji9m8cDP0gX92WsOiRWek9SXPTGVm9wLj\ngXXuPiKy7RjgT0AuUAlc7O7vRd67HrgQqAKucPeXGzq2kiqRzBTPmqorgEVApzgeU1JMYWFhWiUU\noVCIa2++luyh2XxV9RU5JTlM+uUkho8dDkBx8dSAI5QEuh+4E3goatt04Hfu/rKZfQ+4GfgXMxsK\nnAMMAfoCr5jZYQ0tE5GfD+XliQ1eRFJPXJIqM+sLnAb8P+CqeBxTJFGi20LsqllDztActvTYQmfv\nTPec7vzfK//H8OHhpEoLJKcvd3/DzPrV2bwG6Bx53gVYFXl+JvC4u1cBS81sCTAKeKe+Y2umSiQz\nxWumagZwDV8PRiIpqW5biBUbrqbLuZV06NqBI3oeQdnGsm/sn66XPKVB1wFvmtmtgAEnRLb3Af4R\ntd+qyLZ65efDunUJi1FEUlTMSZWZnU64JuFDMysiPBDVa8qUKXufFxUVUVRUFOvpRZrliSdeoqKi\niD1Z7wFgvY9i1bJnGd3pOMq2lIWXn7nmm8vPpNslz+YqKSmhpKQk6DCS5V7gMnf/u5n9ELgPOLm5\nB3n11SksWwZTpmisE2kt4jHWWQMlAU0/gNl/Az8hXLzZHsgHnnH38+vs11D5gUjS/Pznv+KZ+W/T\n9sheVLbbyo49nzHWRzNm9NEAjD9ZHdP3pyUrt6eqyOW/56IK1cvdvVPU+1vcvYuZXQe4u0+LbH8R\nmOzu+1z+MzN/8knn8cfh6aeT9EVEJO5aMtbFPFPl7r8BfhMJ4DvAr+smVCK1YlnmJh5L5HToVgOH\nr6Sqex678pfR5ssaDj20t7qlZy7jm7PrS8zsO+7+mpmNBZZEtj8LPGJmMwhf9hsILGjooJ06qaZK\nJBOpo7okTSzL3MRjiZxQKMSSr5bQqWc22woWcpAdQO8hB8Oer+/yS9f1DGVfZvYoUAR0N7PlwGTg\nP4C7zawNsDvyGndfZGZPEr7DubbVQoNT7ypUF8lMcU2q3P014LV4HlPSRyzL3MS6RE5t64Qd/XZQ\ntrGMNmva0LfvAVQsrGDe51/RseNZQHquZyj1c/cfN/DWsQ3sPxVoUo8NJVUimUkzVZIRZs+ZTdbQ\nLHZ33c3IbSPZ8sYW2pS34ZBuR7Coo5p7SnwpqRLJTEqqJGla2vOptLSUtWtXsXTpf7N9+w46duzQ\n7H5RNV7DsqpldMvtxqCBg1hfvZ7jc49n05qaln4dkQYpqRLJTEqqJGla0vMpupaqR4/hbNhwE9/6\n1jFcfnnTL9G5O1mDs6h6oYpOOZ1Yv2H93tYJFRUVau4pcVebVLmDpcV9kiLSFDG3VGjyidRSQVqg\nuHgq8+ePiro8N5cxYxYwbdr1TT7GC0te4P0173Nal9OY8+oc4JutE+JxV2EmSaeWColQO9a1bQtb\nt0K7dkFHJCItEUhLBZFU9vbKt3l9+etcO/paurTrQuGIfROmTG/uKYlRO1ulpEokc2QFHYBIYyZM\nGEd19QOUlc2lrGxu5PLcuCZ9dvGGxTy16CkuG3UZXdp1SXCkIt+kuiqRzKOZKklpza3DCoVCzJ4z\nm601W1ndazW/Hfdbeuf3Tla4InspqRLJPEqqJOU19fJcbS8qH+osqVxCz5Ke7DlyD3RPQpAideTn\nQ3l50FGISDLp8p+kjdlzZmNDjXWd1zHokEH0OKwHs+fMDjosyVCaqRLJPEqqJG1UezVLq5bStX1X\n+uT3CTocyXBKqkQyj5IqSQvuTuXASmyN0WFtB9Z/FulFdfL4oEOTDKWkSiTzqKZK0sIzi58hr1se\nD//Hw7w0N1zUPv6ar3tRiSRbp05KqkQyjZKqNJVJDS1f/epVPl73MdeOvpYObTow8oiRQYckopkq\nkQykpCoNRS/tAjBv3gxmzmz6si6pbtasWcx8eCYAp5x9CmsL1u5NqERSRX4+bNgQdBQikkxKqtLQ\nE0+8RHb2xKilXcLbmpJUpfoM16xZs7hoykVkj8qmKruK155/jTvOvIPueeqbIKklPx++/DLoKEQk\nmVSoLnvVznDNnz+K+fNHMWnSDEpLS4MOa69QKMQ1N17DrvxdtOnRhppDasjrlMfTTzwddGgi+9Dl\nP5HMo5mqNDRhwjjmzZtBWVn4dXhplyv3+7lYZrgSrbax55b+W6hoV8GaVWsosAK8Sot0S2pSUiWS\neZRUpaHmLu2Saupegmzbti1XT76aZT2Wcejhh/Le2vdgF2x7bRvtt7Vn0pRJAUcssi8lVSKZR0lV\nmmrq0i7RWjrDFU91i+yff34ynQduZrNtZlPNJtaUr+GQnoew5cMtdN3ZlZun3MxZZ52V1BhFmkJJ\nlUjmibmmysz6mtmrZhYys4Vmdnk8ApPkq53hGjNmAWPGLAjkjsHoS5AFBWPZWtGLdZ03MeSkIVTu\nrKS6opqclTmM6jCKvz34NyVU0mJmdq+ZrTOzj6O2PW5mpZHHV2ZWGvXe9Wa2xMwWm9kp+zu+kiqR\nzBOPmaoq4Cp3/9DMOgLvm9nL7v5pHI4tSdaSGa5k2N5lO71H9Kb67WoOsUOYfsN0NfaUWN0P3Ak8\nVLvB3c+tfW5mtwBbIs+HAOcAQ4C+wCtmdpi7N1jUp6RKJPPEnFS5+1pgbeT5djNbDPQBlFRJs9Ve\ngvznP99k3fbZVFV+RcGmDny57EsOyz0M8mH6NUqoJHbu/oaZ9Wtkl3OAosjzs4DH3b0KWGpmS4BR\nwDsNfVhJlUjmiWtNlZkdAoykkYFGWqdk9a8qLCykqKg3M578A1ZotC1ow9pd2zhjxRkcPfhoLT0j\nSWFm3wbWunttp6k+wD+idlkV2dagjh1h506oqYEsNa8RyQhxS6oil/6eAq5w9+317TNlypS9z4uK\niigqKorX6SWBktmhPRQKcf9z95M12sg5JIfdlbvJ35jP9rLtFP+qOO7nk/0rKSmhpKQk6DCS7UfA\nYy39cO1Yl50NL7xQxOmnF8UnKhFJmHiMddZISUDTD2KWA8wGXnD32xvYp7HyA0lhxcVTmT9/VFT/\nqrmMGbOAadOub9HxGpv1mvbHadz6t1vZNngbfpCTtTuL7M+yOdFP5IWnXoj9y0jMzAx3t6DjiIfI\n5b/n3H1E1LZswjNRhe6+OrLtOsDdfVrk9YvAZHffZ1Y+eqzr3Rveew/6NDqnJSKpqCVjXbwmpe8D\nFjWUUInUakrX9oOHHkzljkp8rVP9VTX+kTPpp+pFJQlhkUe0k4HFtQlVxLPAuWbWxsz6AwOBBfs7\neKdOqqsSySTxaKkwGjgPOMnMPojcinxq7KFJqpgwYRzV1Q9QVjaXsrK5kf5V41p0rLotE7KzJ+6d\ntQIYN3Yc29tsp39Bfzov7Eyn0k5MvWyqWidI3JnZo8BbwCAzW25mF0TemkCdS3/uvgh4ElgEPA9c\n3JSpdxWri2SWeNz99yaQHYdYJEUlq0O7u/NOxTv86Ls/ou3nbbFDjPEnB1OYnuoLS0vs3P3HDWy/\noIHtU4GpzTmHkiqRzKKO6tIk8epf1VDXdnfnr4v+yvY92/nt6b8lJyu4v5rJLMyX9KakSiSzKKmS\npGpo1uuVL19h8YbFXDP6mkATKkjthaWldVFSJZJZlFRJ0tWd9Xpv9Xu88uUrXDv6WvJy8wKMTCS+\nlFSJZBYlVZJ0oVCI2XNmA3DEcUfwavmr/Oq4X9GtfbeAIwtLhYWlJT0oqRLJLEqqJKlmzZpF8W3F\nZA3NoluPbvz5yT/zx3/7I3079Q06tL2SVZgv6U9JlUhmUVIlSRMKhSieXszW4VvJ7ZPLyp0rGdRx\nEIvfWcyZx58ZdHjfkKoLS0vrkp8Py5YFHYWIJItWpJKkmT1nNtkHZNOmYxt2td1F+/bt2blhZ9Bh\niSSMZqpEMouSKkmqHv17UL6pHC9zbKVRs6iG8SePDzoskYRQUiWSWZRUSdKc/t3T2VSxiR4H9KDn\nkp50+aQL066aFkhzT5FkUFIlkllUUyVx1Vgn8i9yvuCkfzmJHst7kNMrJ7Bu6SLJoqRKJLMoqZK4\naawT+evLXufd1e9y4/gbyW+bH2icIsmSnw/l5UFHISLJoqRK4qahTuQ5fXJ49rNnuWb0NUqoJKN0\n6qSZKpFMoqRK4m7bthDrts5mx47P+WpzNQ9++CCXjrqUnh16Bh2aSFLp8p9IZlFSJd/QWE3U/kyY\nMI5nnrmKlTu/hJ5G9sE7eSunLbd3vJ3+XfsnKmSRlKWkSiSzKKlKA7EkQnWP01BNVFO0bduWym5L\nyTp6M1l52ThV9OjYl8/f/RxGtygkkVatbVuoqYE9e6BNm6CjEZFEU0uFVq42EZo/fxTz549i0qQZ\nlJaWtuhY0TVRBQVjyc6euDdZa4rZc2aTV5hHwbButB/chryCPPZ8vqdFsYg0lZkdamZtI8+LzOxy\nM+sSdFwAZpqtEskkSqpauVgToXjr1aMX5dvLqdldQ9aWLKrXVau5pyTa00C1mQ0E7gEOAh4NNqSv\nKakSyRxKqgQIz3itW7eWpUv/k2XLZlJWNpfq6geYMGFck49x+ndPZ/PGzRS0LaDH6h50+bQL065V\nc09JuBp3rwJ+ANzp7tcAvQOOaS8lVSKZQzVVrdyECeOYN28GZWXh1+FE6Momfba2Fmvt2lW8//5q\n8vMvoaBgLBs23M5RRw3hiiuaXk8FsKrdKkYXjabXyl7kds1l/KVq7ilJUWlmPwJ+BpwR2ZYbYDzf\noKRKJHPEJakys1OBPxKe+brX3afF47iyf4WFhcyceWVUoXrTEqHoovRly56kvPwUCguP5pBDOtOx\nYwd69VrQrIRqwaoFzFs6jxvH30jX9l1b+nVEWuICYBLw/9z9KzPrDzy8vw+Z2b3AeGCdu4+I2n4Z\ncDFQBfyfu18X2X49cGFk+xXu/nJTglNSJZI5Yk6qzCwLuAsYC6wG3jWzWe7+aazHlqYpLCxs9h1/\n0bVYGzYsoLy8B6tXr6dz585N+nwoFGLabdN4+6O36di3IwefcjB3//RuJVSSdO6+CLgcwMy6AvlN\n/Ifd/cCdwEO1G8ysiPBs1xHuXmVmBZHtQ4BzgCFAX+AVMzvM3X1/J1FSJZI54lFTNQpY4u7L3L0S\neBw4Kw7HlSQ58MBxwEPs3PlWk2qpQqEQ5196Pk+88QRLhy3lo4M/Yt5f5vHuq+8mL2iRCDMrMbNO\nZtYNKAX+x8xu29/n3P0NYHOdzb8EborUaOHukQvrnAU87u5V7r4UWEJ47NsvJVUimSMeSVUfYEXU\n65WRbZLCJkwYR3X1A5SVzaWycjP9+lVyyimljBmzoNHeVKFQiKsnX82nyz6FYZB1WBZtu7elZkAN\nMx+eCYQvLRYXT6W4eGqL2zuINENndy8H/hV4yN2PBb7bwmMNAsaY2dtmNs/MvhXZXnecW0UTxzkl\nVSKZI6mF6lOmTNn7vKioiKKiomSeXqLsW4v1h/1eQgyFQlx787Us67GMyqGVVLapJHtDDlkdHas2\nIPYGopJ6SkpKKCkpCTqMxuSYWW/Cl+d+G+uxgK7ufpyZHQP8FRjQ3INEj3WbNhWxbVtRjGGJSKLF\nY6yzJpQENH4As+OAKe5+auT1dYDXrWkws6aUH0iKqp2hWtZjGX2G9uGNL95g9/rd4YsgfXPILjWm\nX3ET69ZVMH/+qKhFlecyZswCpk27PtgvIHFjZri7BR1HLTP7N+A/gTfd/ZdmNgC42d3PbsJn+wHP\n1Raqm9nzwDR3fy3yeglwHHARgLvfFNn+IjDZ3d+p55jfGOumToWtW+Gmm2L8oiKSVC0Z6+IxU/Uu\nMDAyOK0BzgV+FIfjSoqonaFaXrOcTTWbWLtqLXm5+VSt64AthY5rB9Mj7xTWrasIOlTJQO7+V8Iz\nSrWvvwT2m1BFWORR6+/AScBrZjYIaOPuG83sWeCRSK1WH2AgsKApJ+jUCVas2P9+ItL6xVxT5e7V\nwKXAy0CIcDHn4liPK6lj9pzZ5A7LZejYoVRtq6KyshL/3Om4fCAnDHuN0SPfoFu38OJ+0bVaLWkg\nKtJcZtbXzP5mZusjj6fNrG8TPvco8BYwyMyWm9kFwH3AADNbSLgr+/mw9w7DJ4FFwPPAxU2deldN\nlUjmiEtNlbu/CBwej2NJ6trddTc9R/bEFzjddnSjvF03KirWUlGxdm/T0Zb2zRKJwf2EE6B/i7z+\nSWTbyY19yN1/3MBbP21g/6nA1OYG16lT+PKfiKS/mGuqmnwi1VS1WqFQiEm3T2Jtv7UMzB1I1qIs\npl8znYqKiqjkaVxck6fabu+JOLbEJgVrqj5095H725bEeL4x1n30EZx3HnzySRDRiEhLtWSsU1Il\n9QqFQsyeMxuAI48/kse/eJxea3vRNasr409O7PIzde8grK5+QHcQppAUTKrmEp6Zeiyy6UfABe4+\nNqB4vjHW7dwJ3buHLwHmaGEwkVYjqEJ1STO1hem5w3LZ7bv58+N/5rZ/u43v//j7STl/dLd3gLKy\n8DYlVdKACwl3Rp8BOOE6qYlBBhQtLw969YKlS2HgwKCjEZFEUlIl31DbOmF5zXIOa3cY63LX0aem\nD58t+AxOCDq62OiSYnpy92XAmdHbzOxXhNcjTQmDB8OnnyqpEkl38eioLmkiFApx8eSLed/eZ1nO\nMua+O5e8nXl0z+6e1DgScQdh7SXF+fNHMX/+KCZNmqFu7+ntqqADiFabVIlIetNMVZqJZTbm3r/c\ny6e5n5IzKIeK3RX4emfNC2vo0qcL468Zn6iQ95GIOwh1STHjpEzNF4STqne1NKZI2lNSlUZiWSKm\ntjB9R/8d5LXNo3P7zuxetZuczTlM/+P0hBam16ewsFAJj8Qipe6KGTwYHn446ChEJNGUVKWRls7G\n1BamVx9aTUV5BRXLKyjIKiB7eTbjxzZ8p19rqlGaMGEc8+bNoKws/Lq2r5a0Xma2jfqTJwPaJzmc\nRunyn0hmUFKV4aLX9Cs4soB1y9bBR1BdXs3gvoP5+fk/r/dzrW3hZDUlTT/unh90DE3VsydUV4f/\noVNQEHTQ56HpAAAgAElEQVQ0IpIoSqrSSHNnY2bNmkXx9GI27d5EZZdKVm5YydEDjmZ9xXr6bejH\nLTfc0uAsVWusUdIlRQmK2dezVSeeGHQ0IpIoSqrSSHNmY2bNmsW//+e/s/vw3eS2zWXbjm10WteJ\n9ZvX039Xf6bfkPw6KpF0pqRKJP0pqUpBsdQqNWU2JhQKUTy9mN0jdlN1YBUVVJC/Pp+cD3Po178f\n028IL0FTXDy1wRhUoyTSPKqrEkl/SqpSTDJqlWbPmU32Adl07NaRje02YnuMqh1VHNDuAG654RYq\nKir2G4NqlESaZ/BgeP31oKMQkURSUpViklWr1KN/D1ZtXkXHNh2pXFtJu8/aMe3GaQwbNozi4qlN\nikE1SiJNp5kqkfSnpCqD1PaiWrtmLavLV9OrZy+y/plFzboapt04jbPOOivoEEXS1oABsHIl7N4N\n7doFHY2IJIKSqhSTqFql6EWSV3dZTdWmKs6wM/BDnR1ds3jrrUUcdNBBFBYWql5KJAFyc6F/f/j8\ncxg+POhoRCQRzD05jYfNzJN1rtauvkL1WBttTvvjNP5R+Q+qDqxizfY19N7amwEbBvD2a5v31k5V\nVz+wt3aqNTX2lOQyM9w9pZaBSSWNjXU/+AGcdx788IdJDkpEmq0lY52SqhTRWBJTt3g9OvnZn9pL\nfiVvlFDWv4zKQys58oAj2fr5VjbP2c2ebddE1U7NZcyYBUybdn3cv5+kDyVVjWtsrLv+eujQAX73\nuyQHJSLN1pKxTpf/UsD+7vhrafH6rFmzKL6tmKyhWeQeksuStUsY6SPZum0rlaFKBh40mEWLEvjF\nROQbBg+GOXOCjkJEEiUrlg+b2XQzW2xmH5rZ02bWKV6BZZLopKmgYCzZ2RP3zlq1VG0vqq2Dt7Kt\nzza+avMVA3sNpPPSzhyfezzTr5nOJZecT3X1A5SVzaWsbG6kdmpcXL6TSKozs3vNbJ2ZfRy1bbKZ\nrTSz0sjj1Kj3rjezJZEx75SWnFN3AIqkt1hnql4GrnP3GjO7Cbg+8pA4aknh+L0P3cum3ZvYs2wP\nHAB5eXlUraqi6MQiin9VvHc/9ZqSDHY/cCfwUJ3tt7n7bdEbzGwIcA4wBOgLvGJmhzW3puHww+Gz\nz8A9vHSNiKSXmJIqd38l6uXbwNmxhZOZ9pc0NbfRZigU4vm3n6fmiBp2tNuBlRo57XOoWV/D+EvH\nf2Nf9ZqSTOXub5hZv3reqi/dOQt43N2rgKVmtgQYBbzTnHN26QIdO8KqVdC3b/NjFpHUFs+aqguB\nx+N4vIzRlKSpqclPKBTi6slXs2fAHvYU7KFju45UbqzEPjKm3TxN6/mJ7N+lZvZT4D3g1+6+FegD\n/CNqn1WRbc1WewlQSZVI+tlvUmVmc4ADojcBDvzW3Z+L7PNboNLdH23sWFOmTNn7vKioiKKiouZH\nnKbiMWNUW5i+kY3s7r0bM6Ov96W6oJpTzz61VTX3VEuH1FFSUkJJSUnQYSTL3cDv3d3N7A/ArcC/\nN/cgjY11tUnVd78bc6wiEkfxGOtibqlgZhOBi4CT3L2ikf3UUiGBQqEQZ//H2WwdvpWKjhVsW72N\nzu0707NtT/rv6s/0a6a3mlmqWFpISOKlU0uFyOW/59x9RGPvmdl1gLv7tMh7LwKT3X2fy3/7G+tu\nvx2WLIG77orb1xCRBGjJWBfr3X+nAtcAZzaWUEli1V7y27R7E5VeSdYBWXTv2Z2sL7Lot6Ffq0qo\nIDF3Q4o0wIiqoTKzXlHv/SvwSeT5s8C5ZtbGzPoDA4EFLTmh7gAUSV+x1lTdCbQB5lj4Vpa33f3i\nmKOSJqu95Lep5yZ2j9jNzm076f5pdyzX6EY3brnhlnoTKl1ek0xnZo8CRUB3M1sOTAb+xcxGAjXA\nUuAXAO6+yMyeBBYBlcDFLZ16V1Ilkr7UUb0Vi77kV1NQw8ZdG8nblEe7he3o1q4b066tf5HkVL+8\nlurxZbp0uvyXCPsb62pqID8f1q4N/xSR1JT0y38SnFmzZvGDn/2AFWtWsKd6D1Wdq+iW1412O9rx\nrf7f4ul7nm6wMD3VL6/V3g05ZswCxoxZoIRK0kpWFgwaFO5XJSLpRcvUtEKzZs3ioikXsWvYLiq2\nVrBz4046fdWJNjVt6Ly+M7fcXf8lv9ZE/bMkndVeAjz66KAjEZF40kxVKzTz4Zlkj8qmx1E9yBqZ\nRVaHLKpeqqLzJ52ZdtX+e1FNmDBOy9OIBGjIEFi4MOgoRCTeVFPVCn3vh9/jg+4fwGFQXVXNrk92\nceCnB/K3B//W5BkqFapLS6mmqnFNGeveew/OPTfcWkHL1YikppaMdUqqWqG///3vnP+/58MgyNuR\nR82CGv5nyv+0quae0nopqWpcU8Y69/AlwIcegmOPTVJgItIsSqrSWCgUYvac2QB0G9GNV5e8ysZX\nNpLt2Uz66SQlVJI0Sqoa19Sx7sYbYf16uPPOJAQlIs2mpCpN1faiyhqaRYceHdi0bRMPn/8wJxx1\nQtChSQZSUtW4po51X34Jxx0XXlw5NzcJgYlIs6ilQhoKhUIUTy9m6+CtbO2zlc+qPqNXfi9ef+31\noEMTkRgMGAADB8KcOUFHIiLxoqQqxc2eM5vsA7LJzs9md5vd5Ofls3nD5qDDEpE4+MlP4C9/CToK\nEYkXJVWtQLf+3SgvL6dNWRtqVtRQs6iG8SePDzosEYnROefA88/Dtm1BRyIi8aCkKsWdfNLJrKte\nR/+u/en2ebcm96ISkdRXUADf/jb8/e9BRyIi8aBC9RRUe6dftVdT3r+cbm27UfNpDWbG+JPHM2zY\nMPWZksCoUL1xzR3rnngC7rsPXkqdlaJEBN39lxZCoRDX3nwtOUNzWF69nMrVlTz2i8c4YvgRe/cp\nLS3l/PP/wKZN4TYK3brN4qGHfqfESpJCSVXjmjvW7dwJffrA4sXQq1cCAxORZtHdf61cKBTi6slX\n81X7ryjLL6NNQRsGHDSA5195/hv73XHHgyxdegq7dp3Grl2nsXTpKdxxx4NJjbW0tJTi4qkUF0+l\ntLQ0qecWSSd5eXDWWfD440FHIiKxUlKVImbNmsXZ/3E273/1PqtrVvPJik84MPdAsmzfP6LFi78C\netCmTfgBPSLbkqO0tJRJk2Ywf/4o5s8fxaRJM5RYicRAdwGKpAclVSkgFApRfFsxW4dvxQud7eXb\nyVmTw2fvfEZlqHKfO/0GDz4YeIg9e+ayZ89c4KHItuR44omXyM6eSEHBWAoKxpKdPXFvfZeINN+/\n/AusXg2ffhp0JCISCyVVKWD2nNlkDc0i+6BsqgdW0y2vG9kfZ9NvQz+mXzN9nzv9rrjiQvr1q6R9\n+ydp3/5J+vWr5IorLgwoehGJVXY2XHghTJsWdCQiEoucoAOQsO49urNy50o65nXEc5zO7Tpzyw23\n1Ns6obCwkIcf/kPU3X+/SGqR+oQJ45g3bwZlZeHX1dUPMGHClUk7v0g6uvZaGDIE3n47vHyNiLQ+\nuvsvBSz4cAHnPXAe+Z3y2b1hNzWLaph21bSUXiRZLR0yl+7+a1wsY91f/gJ//CO880549kpEghNY\nSwUz+zVwM1Dg7psa2EdJVT32VO/htn/cRt7OPHYt2gWwtxeVSCpKl6TKzO4FxgPr3H1Enff2GdPM\n7HrgQqAKuMLdX27guC0e69zDzUB/9jO46KIWHUJE4iSQpMrM+gL/CxwOfEtJVdPVeA0z35tJ+5z2\nTBw5EbNW//8pyQBplFSdCGwHHopOquob08xsCPAocAzQF3gFOKy+QS3Wse6DD+B73wv3reratcWH\nEZEYBdWnagZwTRyOk1Hcncc/eZyKqgp+euRPlVCJJJm7vwHUtzp5fWPaWcDj7l7l7kuBJcCoRMR1\n1FHwr/8K//VfiTi6iCRSTIXqZnYmsMLdFyopaJ6XvniJLzZ9wdUnXE1OVviPQXVKIsFqZEzrA/wj\n6vWqyLaEuPFGGDoU/v3f4cgjE3UWEYm3/SZVZjYHOCB6E+DA74DfACfXea9BU6ZM2fu8qKiIoqKi\npkeaRt5Z+Q6vLX2N4hOLaZ/bHvi6oWZ29kQA5s2bwcyZV8acWClRk1iVlJRQUlISdBgJZ2bt2XdM\na5FYx7ru3eGGG+Cyy+C110D/ZhVJvHiMdS2uqTKz4YTrCnYSTqb6Ev7X2yh3X1/P/hlbU1W7QDLA\n0GOH8tq217jq+Ks4MP/AvfsUF09l/vxRFBSMBaCsbC5jxixg2rTrW3zeuoladfUDcUnUJLOlS00V\ngJn1A55z9xGNjWmEC9Rx95sin3sRmOzu79RzzLiMddXVcOyxcMEFcMklMR9ORJqpJWNdiy//ufsn\nwN7lP83sK6DQ3eurUchYtQsk5w7LZVfNLv781z9z5zl3fiOhSpTozucAZWXhbUqqRPayyKPRMc3M\nngUeMbPbCF/2GwgsSGRg2dnh9QBPOCFcZ3XCCYk8m4jEQzw7qjv7ufyXie596F6Wli9lxRcrWGbL\nOKjnQXzy9if77Ddhwjiqqx+grGwuZWVzIw01xwFavFgkEczsUeAtYJCZLTezC+rssndMc/dFwJPA\nIuB54OJkTL0PHAj33w/nnANr1iT6bCISKzX/TKBQKMTZF5/NliFb2N11N22+asPwPsMZd9A4in9V\nvM/+9dU/xXIJT5f/JBHS6fJfIiRirPv97+Hll+HVV6FNm7geWkQaEFjzzyadKAOTqml/nMaLm17k\ng8oPyM3LJXtlNl0+6cLT9zzd5OaesdZaqVBd4k1JVeMSMdbV1MD3vw8HHwx33RXXQ4tIA5JaUyX7\n5+5sab+FQ3seSnZ5NtuztnPqmFOT2i29sLBQiZRIK5eVBQ8/DMccAw8+GO64LiKpR0lVArUb1o5d\n/7eLQdmDyMrNonJXJT+/9OfNOoYWLxYRgM6d4e9/h6Ii6N8fxowJOiIRqUuX/xJk3lfzmLd0Hmd0\nO4NX570KtHxNP13Ck1Siy3+NS/RY9+qrcO65MGsWHH98wk4jkvFUU5UiPljzAY998hjXjr6WgryC\noMMRiSslVY1Lxlj34otw/vnwf/8XviQoIvGnpCoFfLn5S/604E9cfuzl9OvSD0jeTJNmtCQZlFQ1\nLllj3XPPhZexeeklGDky4acTyThKqgK2bvs6bnnrFn428mcM7zkcSF5bA7VPkGRRUtW4ZI51Tz8N\nl14Kc+bA8OFJOaVIxmjJWBfP5p8ZrbyinDveuYOzBp+1N6ECuP32+1i5Mo8NGxaQm9uV7OyJe2eT\n4im6e3pBwdiEnUdEUsfZZ8Ntt8Epp8C77wYdjYjo7r84qKiq4K4Fd3Fc3+M48eAT924vLS3l5ZcX\ns23bxeza1YWyshkcfPDoACMVkXTzox9BXh6cdhr87//CWWcFHZFI5lJSFaMar+Ge9++hT34fxg8a\n/433nnjiJQoKrmDXroOAvlRVfZ8NG25lwoT4d+9T6wWRzHXWWdCnT/jn0qVwxRVBRySSmZRUxcDd\neXThozjOT0b8BLN9L7127NiBESMGsHr1enbu3M4ppxyTkDqnwsJCZs68MqpQXfVUIpnk6KPhrbfg\n9NPhiy9gxozwoswikjwqVI/B80uep3RNKVefcDXtctrt837QxeN17wYEdHegxEyF6o0LeqzbuhV+\n+MPwGoEPPQTduwcWikirprv/kuitFW8x+5+zKR5dTOd2nRvcL6g2B3UTuvLyWzFrR37+JYDuDpSW\nU1LVuFQY6yor4Te/gSeegL/8Rd3XRVpCa/8lyaINi3hm8TP8+vhfN5pQQXBr70XfDQiwbNmTwAn0\n71+7MHN4HyVVIuknNxduvhlOOgkmTIBJk+B3v9PlQJFEU0uFZlqxdQX3lt7LpKMn0Tu/d9DhiIg0\n6Hvfg9JSmD8fxo6FlSuDjkgkvSmpaoaNOzdy14K7OG/EeQzsNjDocBo1YcI4qqsfoKxsLmVlc+na\ndQXdus3a+zp8d+C4oMMUkQTr3Rtefjncy+qoo+BPf4Lq6qCjEklPqqlqoh17dnDzWzfz7YO/zdgB\nY4MOp0lUqC6JoJqqxqXyWLdoEfziF7BnD/z5z1reRqQxKlRPkMrqSm5/53b6de7Hvw37t6DDEQmU\nkqrGpfpYV1MD998fLmT/yU/ghhugY8egoxJJPYEsU2Nml5nZYjNbaGY3xXq8VOPu3P/h/XRu25kf\nDv1h0OGISJyY2b1mts7MPo7a9nsz+8jMPjSzV8ysb9R715vZksh4d0owUccuKwt+/nP45JPwDSuH\nHRa+JLhnT9CRibR+MSVVZlYEnAEc4e5HALfEI6hU8tSipyivKGfiyIn1NvcUkVbrfqBuYeF0dz/S\n3UcCs4DJAGY2FDgHGAJ8D7jbWvmA0KMHPPggPP88zJ4NgweH2y+o3kqk5WKdqfolcJO7VwG4e1ns\nIaWOuV/OJbQhxC+P/iW52blBhyMiceTubwCb62zbHvWyA7Ax8vxM4HF3r3L3pcASYFQy4ky0o46C\nF14IXxK8++5wndVf/wpVVUFHJtL6xJpUDQLGmNnbZjbPzI6OR1Cp4P3V7/PyFy9z2ajL6NCmQ9Dh\niEiSmNkfzGw5MBGYGtncB1gRtduqyLa08Z3vwJtvwn//N9x+e/iy4B13wPbt+/+siITtN6kyszlm\n9nHUY2Hk55mEm4d2dffjgGuBJxMdcDIs2biExz55jEtHXUr3PK3xIJJJ3P137n4w4cuDfww6nmQy\ngzPOgDfegMceg9dfh0MOgeuuCy/ULCKN229HdXc/uaH3zGwS8Exkv3fNrMbMurv7xvr2nzJlyt7n\nRUVFFBUVNTfehFuzbQ1/fv/PXHjUhRzU+aCgwxEJXElJCSUlJUGHEYRHgecjz1cB0QNC38i2erWG\nsW5/jjsufBnwyy/hzjvhmGPgyCPhwgvhBz+A9u2DjlAkvuIx1sXUUsHM/gPo4+6TzWwQMMfd+zWw\nb0rfZgywdfdWpr05jTMGncHxBx0fdDgiKSmdWiqY2SHAc5EbbTCzge7+eeT5ZcAod/9ppFD9EeBY\nwpf95gCH1TeotYaxriV274Znn4X77oN334VzzoFzz4UTT9TyN5Kekt6nysxygfuAkUAF8Gt3f62B\nfVN6oNldtZtb3rqFwt6FnHbYaUGHI5Ky0iWpMrNHgSKgO7CO8J1+pwOHA1XAl8Av3X19ZP/rgZ8D\nlcAV7v5yA8dN6bEuHpYvh4cfhqeegtWrwzNXP/xhuC4rV/f0SJpQ888Wqq6p5k/v/olu7btx3hHn\nqXWCSCPSJalKlFQe6xLhiy/g6afDj88/h+9+F8aNCz/6pFUpv2QaJVUt4O489NFDbNuzjYuPuZgs\n03KIIo1RUtW4VB3rkmHVqvA6gy++CK+8El538JRT4NvfhtGjoWfPoCMUaTolVS3w3GfPsXD9Qn59\n/K9pm9M26HBEUp6Sqsal6liXbNXV8N57MHdu+G7Ct96CXr3CNVgnnABHHw1Dh0LOfm+XEgmGkqpm\nemP5G7yw5AWKTyymU9tOQYfTInUXTdYiyZJoSqoal4pjXSqorg4vjfP66/D22/D+++HarBEj4Fvf\nCjcdHT4chg2D/PygoxVRUtUsn6z/hAc/fJCrT7iaAzoeEHQ4LVJaWsqkSTPIzp4IQHX1A8yceaUS\nK0koJVWNS7WxLpWVl8MHH4QTrI8+glAIFi8OL6EzfDgcfjgMGhRuRDpoEBx4YHjtQpFkaMlYl5ET\nr8u2LOP+D+7nklGXtNqECuCJJ14iO3siBQVjgfDiqE888ZKSKhFpFTp1Ct8x+J3vfL2tuhq++io8\nq/XZZ+H2DY8+Cv/8ZzgJ69cv3JA0+mffvuGi+AMPhLaq4pAAZVxSVbazjD+9+yd+euRPGdB1QNDh\niIhIlOxsGDgw/Khr+/ZwZ/fax7Jl4bqtlSvDRfJr10LnzuEE64AD9n107w4FBV//7NQp3EVeJF4y\nKqmq8Rr+tOBPnHbYaYzsNTLocGI2YcI45s2bQVlkGevq6geYMOHKYIMSEUmQjh3DlwWHD6///Zoa\n2LAhnGCtW/f1Y/Vq+PBD2LgxPKNf+3PXrnAS1qULdO0a/tmlSzjZ6tQp/F6nTuEar44dv3506BB+\n5OV9/bN9ezVBlQysqdqwYwM9OvQIOoy4UaG6JJtqqhqXKmOd7F9lJWzZAps3f/1z69bwZcby8q+f\nb9/+zce2bbBz59ePHTvCCVpuLrRrF06w2rcPP699tG379c82bb7+WfvIzQ0/op/n5obvjox+XvvI\nzm74Z+0jK2vf59E/G3qY7fvc7JvP63uvsUdrpEJ1EUk4JVWN01iXmdyhoiK8nM+uXeHH7t1fb4v+\nuWdP+FH7vKIinODVPvbsCf+sqvrm9urq8LboR3X119ujf9bUfP1e7evabbU/3b/eXneb+9evo7fV\n/mxoW91HXfUlW409r7ut7vaGttX3Xt04on/+7Gdw661191FSJSIJpqSqcRrrRPbVUMLV2PO62+pu\nb2hbfe/VjaXue23b7tvKQ3f/iYiISMppzZcBm0MdP0RERETiQEmViIiISBwoqRIRERGJAyVVIiIi\nInGgpEpEREQkDpRUiYiIiMSBkioRERGROFBSJSIiIhIHMSVVZnaMmS0wsw8iP4+OV2AiIolkZvea\n2Toz+zhq23QzW2xmH5rZ02bWKeq9681sSeT9U4KJWkRSWawzVdOB37n7UcBk4ObYQ0q8kpKSoEMA\nUicOUCz1SZU4ILViSSP3A+PqbHsZGObuI4ElwPUAZjYUOAcYAnwPuNssfftDt/a/b609fmj936G1\nx99SsSZVa4DOkeddgFUxHi8pUuUPO1XiAMVSn1SJA1IrlnTh7m8Am+tse8XdayIv3wb6Rp6fCTzu\n7lXuvpRwwjUqWbEmW2v/+9ba44fW/x1ae/wtFevaf9cBb5rZrYABJ8QekohISrgQeCzyvA/wj6j3\nVkW2iYjstd+kyszmAAdEbwIc+B1wGXCZu//dzH4I3AecnIhARUSSxcx+C1S6+2P73VlEJMLcveUf\nNit39+hCzq3u3rmBfVt+IhFJKe6eFvVEZtYPeM7dR0RtmwhcBJzk7hWRbdcB7u7TIq9fBCa7+zv1\nHFNjnUiaaO5YF+vlvyVm9h13f83MxgL/jFdgIiJJYJFH+IXZqcA1wJjahCriWeARM5tB+LLfQGBB\nfQfUWCeSuWJNqn4B/MnM2gC7gf+IPSQRkcQzs0eBIqC7mS0nfAfzb4A2wJzIzX1vu/vF7r7IzJ4E\nFgGVwMUeyzS/iKSlmC7/iYiIiEhY0juqm9llkeZ5C83spmSfv04svzazGjPrFmAMDTYbTNL5TzWz\nT83sn2ZWnMxz14mjr5m9amahyN+Ny4OKJRJPlpmVmtmzAcfR2cz+Gvk7EjKzYwOM5fpIDB+b2SOR\nGWqJkiq/T83RQBPUrmb2spl9ZmYvmVm9tbKpoKGxo7V8BzNra2bvRJpoh8zsvyPbW0X8teqOma0w\n/qVm9lFtM/PItmZ/h6QmVWZWBJwBHOHuRwC3JPP8dWLpS/hOxWVBxRBRb7PBZDCzLOAuwg0QhwE/\nMrPByTp/HVXAVe4+DDgeuCTAWACuIHypJ2i3A8+7+xDgSGBxEEFECrovAo6KFHXnAOcGEUuqSrHf\np+aorwnqdcAr7n448CpJHJdaoKGxo1V8h0jt3r9EmmiPAE4ys9G0kvij1B0zW1v8NUCRux/l7rU9\n6Jr9HZI9U/VL4CZ3rwJw97Iknz/aDMIFqYFqpNlgMowClrj7MnevBB4Hzkri+fdy97Xu/mHk+XbC\nyUMgfYAiCfdpwP8Gcf6oODoB33b3+wEijSfLAwqnHNgDdDCzHCAPWB1QLKkqZX6fmqO+JqiE434w\n8vxB4PtJDaoZGhg7+tK6vsPOyNO2hP+/vJlWFH8DY2ariT/C2DcnavZ3SHZSNQgYY2Zvm9k8C2it\nQDM7E1jh7guDOH8jLgReSOL5+gArol6vJAUaGprZIcBIYJ/b1ZOkNuEOuuCwP1BmZvdHptXvMbP2\nQQTi7puBW4HlhBtfbnH3V4KIJYWl5O9TC/V093UQTlqAngHH0yRRY8fbwAGt5TtELp19AKwFStx9\nEa0ofuofM1tT/BCOfY6ZvWtm/x7Z1uzvEOvdf/uwxpuF5gBd3f04MzsGeBIYEO8YmhDHb/hmk9KE\n3gLdSCy/dffnIvvUNht8NJGxpDoz6wg8BVwR+Vdnss9/OrDO3T+MXK4O8vb4HKAQuMTd3zOzPxKe\njp6c7EDMbABwJdAP2Ao8ZWY/zvS/rxkk6H9g7FfdscP27ReWst8hcrXiqMjs9EuRsadVxF/PmNmQ\nlIw/ymh3X2NmPYCXzewzWvBnEPekyt0b7KhuZpOAZyL7vRspEu/u7huTFYeZDQcOAT4yMyM8Tfy+\nmY1y9/XxjqOxWKJimkh46vSkRJy/EauAg6Ne9yXA9Rsjl5WeAh5291kBhTEaONPMTgPaA/lm9pC7\nnx9ALCsJz6i+F3n9FBBU8fPRwJvuvgnAzJ4hvCyVkqqvpdTvU4zWmdkB7r7OzHoBCRkb46WBsaNV\nfQcAdy83s+cJ/761lvjrGzMfBta2kvgBcPc1kZ8bzOzvhC/nN/vPINmX//5OJHEws0FAbiISqsa4\n+yfu3svdB7h7f8L/4zoqUQnV/tjXzQbPrNNsMBneBQaaWb/InVznEm5yGJT7gEXufntQAbj7b9z9\nYHcfQPi/x6sBJVREpp1XRH5XAMYSXPH8Z8BxZtYu8o+RsQRUNJ/CUu33qTm+0QSVcNwTI89/BgT1\nj5ymqm/saBXfwcwKau8qi1zePxn4gFYSfwNj5k+B52gF8QOYWV5kphMz6wCcAiykBX8GcZ+p2o/7\ngfvMbCFQAQTyP6s6nGAv8dxJPc0Gk3Fid682s0sJ34GYBdzr7kHdXTYaOA9YGKktcOA37v5iEPGk\nkMsJd/LOBb4ELggiCHf/yMweAt4HqgkP+vcEEUuqSqXfp+aw+pug3gT81cwuJHyH9DnBRdi4hsYO\nYMjnsCMAAAPrSURBVBrwZCv4Dr2BByP/WMkiPNs2N/JdWkP8DbmJ1hP/AcDfIpeMc4BH3P1lM3uP\nZn4HNf8UERERiYOkN/8UERERSUdKqkRERETiQEmViIiISBwoqRIRERGJAyVVIiIiInGgpEpEREQk\nDpRUiYhIyjOzbmb2QWQdzDVmtjLqdZN6LprZvWZ22H72udjMfhSfqOs9/g+iGvpKmlGfKhERaVXM\n7L+A7e5+Wz3vmafw/9giS7g8FeBSXJJAmqkSEZHWZu8qGGZ2qJmFzOwvZvYJ0MvM/mxmC8xsoZn9\nLmrf181shJllm9lmM5tqZh+a2ZtmVhDZ50Yzuzxq/6lm9o6ZLTaz4yLb88zsKTP7xMz+ambvmtmI\nfYI0uzkS24eR45xIeJ3X2yIzbAeb2UAzezFyjBIzGxj57MNmdreZvWdmn0aWNMPMhke+W2nkuIck\n7L+yNFuyl6kRERGJt8OBn7j7BwBmVuzuW8wsG5hnZk+5+6d1PtP5/7d3PyE2hWEcx79PjSjTzE6p\nKUmjkAZNkSzsNAs1CyJ2xEJRZmehrMmGFHYypUwmicnInw1WxmKKKRqymGah5H8NM34W9xmuM/di\n6tTMrd+nTr333Pd9z1k+Pc/beYAHko5FxGlgH3Cy1uaSNkbEdiotfLqAw8C4pB0ZTA0V10TEEqBL\n0pr83VLVMLlP0o28fx/YL+l1RGwGzgHbcps2SZ1ZLrwbESuAQ8ApSX3Zvmou26xZgYMqMzNrdKPT\nAVXam/3amqj01lsNFIOqr5Lu5HgI2FJn7/6qOctyvIVKbzskDUfEsxrr3gFTEXERGABuFidkI+VN\nwLXs/Qd/VpCu5jNeZF/GduAxcDwzVP2SRuu8t80Bl//MzKzRfZkeZPnsCLBVUgcwCCyqseZb1XiK\n+kmGif+YMyNbJGkS6ASuA93ArTrr3kraIGl9Xh3V2xTmSlJv7jcB3M6Sos0TDqrMzKzRVQc1LcBH\n4HNELOV3Ke1va2brEbALICLWAqtmbB7RDLRKGgB6gHX516d8RyS9B8YjojvXROFs1s68vxJoA15G\nxHJJrySdoZL9mnGWy+aOy39mZtbofmV0JD2NiBFgBHgDPKw1rzD+574FZ4FLeTD+eV4fCnNagf6I\nWEglgDua968AFyKih0rGaTdwPiJOAAuAXmA4545FxBNgMXBA0mRE7MlPPnwHxqic87J5wp9UMDMz\nm4U8AN8kaSLLjYNAu6QfJT7jMlUH2q0xOFNlZmY2O83AvaqPjh4sM6BKzng0IGeqzMzMzErgg+pm\nZmZmJXBQZWZmZlYCB1VmZmZmJXBQZWZmZlYCB1VmZmZmJXBQZWZmZlaCn3k/n05X32zbAAAAAElF\nTkSuQmCC\n", + "text/plain": [ + "<matplotlib.figure.Figure at 0x7f8047207c10>" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "#@test {\"output\": \"ignore\"}\n", + "import tensorflow as tf\n", + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "\n", + "# Set up the data with a noisy linear relationship between X and Y.\n", + "num_examples = 50\n", + "X = np.array([np.linspace(-2, 4, num_examples), np.linspace(-6, 6, num_examples)])\n", + "# Add random noise (gaussian, mean 0, stdev 1)\n", + "X += np.random.randn(2, num_examples)\n", + "# Split into x and y\n", + "x, y = X\n", + "# Add the bias node which always has a value of 1\n", + "x_with_bias = np.array([(1., a) for a in x]).astype(np.float32)\n", + "\n", + "# Keep track of the loss at each iteration so we can chart it later\n", + "losses = []\n", + "# How many iterations to run our training\n", + "training_steps = 50\n", + "# The learning rate. Also known has the step size. This changes how far\n", + "# we move down the gradient toward lower error at each step. Too large\n", + "# jumps risk inaccuracy, too small slow the learning.\n", + "mu = 0.002\n", + "\n", + "# In TensorFlow, we need to run everything in the context of a session.\n", + "with tf.Session() as sess:\n", + " # Set up all the tensors.\n", + " # Our input layer is the x value and the bias node.\n", + " input = tf.constant(x_with_bias)\n", + " # Our target is the y values. They need to be massaged to the right shape.\n", + " target = tf.constant(np.transpose([y]).astype(np.float32))\n", + " # Weights are a variable. They change every time through the loop.\n", + " # Weights are initialized to random values (gaussian, mean 0, stdev 1)\n", + " weights = tf.Variable(tf.random_normal([2, 1], 0, 0.1))\n", + "\n", + " # Initialize all the variables defined above.\n", + " tf.initialize_all_variables().run()\n", + " \n", + " # Set up all operations that will run in the loop.\n", + " # For all x values, generate our estimate on all y given our current\n", + " # weights. So, this is computing y = w2 * x + w1 * bias\n", + " yhat = tf.matmul(input, weights)\n", + " # Compute the error, which is just the difference between our \n", + " # estimate of y and what y actually is.\n", + " yerror = tf.sub(yhat, target)\n", + " # We are going to minimize the L2 loss. The L2 loss is the sum of the\n", + " # squared error for all our estimates of y. This penalizes large errors\n", + " # a lot, but small errors only a little.\n", + " loss = tf.reduce_mean(tf.nn.l2_loss(yerror))\n", + "\n", + " # Perform gradient descent. \n", + " # This essentially just updates weights, like weights += grads * mu\n", + " # using the partial derivative of the loss with respect to the\n", + " # weights. It's the direction we want to go to move toward lower error.\n", + " update_weights = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)\n", + " \n", + " # At this point, we've defined all our tensors and run our initialization\n", + " # operations. We've also set up the operations that will repeatedly be run\n", + " # inside the training loop. All the training loop is going to do is \n", + " # repeatedly call run, inducing the gradient descent operation, which has the effect of\n", + " # repeatedly changing weights by a small amount in the direction (the\n", + " # partial derivative or gradient) that will reduce the error (the L2 loss).\n", + " for _ in range(training_steps):\n", + " # Repeatedly run the operations, updating the TensorFlow variable.\n", + " sess.run(update_weights)\n", + " \n", + " # Here, we're keeping a history of the losses to plot later\n", + " # so we can see the change in loss as training progresses.\n", + " losses.append(loss.eval())\n", + "\n", + " # Training is done, get the final values for the charts\n", + " betas = weights.eval()\n", + " yhat = yhat.eval()\n", + "\n", + "# Show the results.\n", + "fig, (ax1, ax2) = plt.subplots(1, 2)\n", + "plt.subplots_adjust(wspace=.3)\n", + "fig.set_size_inches(10, 4)\n", + "ax1.scatter(x, y, alpha=.7)\n", + "ax1.scatter(x, np.transpose(yhat)[0], c=\"g\", alpha=.6)\n", + "line_x_range = (-4, 6)\n", + "ax1.plot(line_x_range, [betas[0] + a * betas[1] for a in line_x_range], \"g\", alpha=0.6)\n", + "ax2.plot(range(0, training_steps), losses)\n", + "ax2.set_ylabel(\"Loss\")\n", + "ax2.set_xlabel(\"Training steps\")\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "lSWT9YsLP1de" + }, + "source": [ + "This version of the code has a lot more comments at each step. Read through the code and the comments.\n", + "\n", + "The core piece is the loop, which contains a single `run` call. `run` executes the operations necessary for the `GradientDescentOptimizer` operation. That includes several other operations, all of which are also executed each time through the loop. The `GradientDescentOptimizer` execution has a side effect of assigning to weights, so the variable weights changes each time in the loop.\n", + "\n", + "The result is that, in each iteration of the loop, the code processes the entire input data set, generates all the estimates $\\hat{y}$ for each $x$ given the current weights $w_i$, finds all the errors and L2 losses $(\\hat{y} - y)^2$, and then changes the weights $w_i$ by a small amount in the direction of that will reduce the L2 loss.\n", + "\n", + "After many iterations of the loop, the amount we are changing the weights gets smaller and smaller, and the loss gets smaller and smaller, as we narrow in on near optimal values for the weights. By the end of the loop, we should be near the lowest possible values for the L2 loss, and near the best possible weights we could have." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "dFOk7ERATLk2" + }, + "source": [ + "## The details\n", + "\n", + "This code works, but there are still a few black boxes that are worth diving into here. `l2_loss`? `GradientDescentOptimizer`? What exactly are those doing?\n", + "\n", + "One way to understand exactly what those are doing is to do the same thing without using those functions. Here is equivalent code that calculates the gradients (derivatives), L2 loss (sum squared error), and `GradientDescentOptimizer` from scratch without using those functions." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "cellView": "both", + "colab": { + "autoexec": { + "startup": false, + "wait_interval": 0 + }, + "output_extras": [ + { + "item_id": 1 + } + ] + }, + "colab_type": "code", + "collapsed": false, + "executionInfo": { + "elapsed": 657, + "status": "ok", + "timestamp": 1446499870301, + "user": { + "color": "", + "displayName": "", + "isAnonymous": false, + "isMe": true, + "permissionId": "", + "photoUrl": "", + "sessionId": "0", + "userId": "" + }, + "user_tz": 480 + }, + "id": "_geHN4sPTeRk", + "outputId": "85c49bf6-8d07-401a-ae08-79c6933adff5" + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlUAAAEPCAYAAABr+zG+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl8VPXZ///XlYUlLEEIIIsCCqKAqCmgFkvTUkUrLnft\nXay2va12obZK9RZTWr8/oO19K2pLXWqxLWqxWnGrWEpVVCK3K2pcBxSs7MgSdgiELNfvj5ngGJKQ\nZCZzZnk/H4/zyJkzZ865RuDjlc/5fK6PuTsiIiIiEpusoAMQERERSQdKqkRERETiQEmViIiISBwo\nqRIRERGJAyVVIiIiInGgpEpEREQkDuKSVJnZFDMLmdm7ZvaAmbWJx3VFRFqLmc02s01m9m7UseFm\n9rKZvWNm88ysY9R7U8xshZktM7OzgolaRJJZzEmVmfUDvg+c4u7DgRzg4livKyLSyu4FxtU59mfg\nenc/Cfg7cD2AmQ0BvgGcAJwD3GVmlsBYRSQFxKOnahdwAOhgZjlAHrAhDtcVEWk17v4isL3O4UGR\n4wDPAhdF9s8HHnL3KndfBawARiUkUBFJGTEnVe6+HfgNsAZYD+xw92djva6ISABCZnZ+ZP8bQN/I\nfh9gbdR56yPHREQOisfjv2OAa4B+QG+go5ldEut1RUQCcDnwYzN7HehAuBdeRKRJcuJwjRHAS+6+\nDcDMHgc+DzwYfZKZaZFBkTTh7mk5nsjdlxMZZ2Vmg4BzI2+tB46KOrVv5Ngh1NaJpI/mtnXxGFP1\nIXCambWLDNwcCyxrILik2KZOnRp4DMkUh2JJ7jiSLZY0Y5Et/MKse+RnFnADMCvy1pPAxWbWxswG\nAAOBJQ1dNOg/o3T6+5aJ8afDd0j1+N1b1tbF3FPl7u+Y2RzgTaAaeAv4Y6zXFRFpTWb2IFAEdDOz\nNcBUoJOZ/Rhw4HF3vw/A3Zea2cPAUqASuNJb2uqKSNqKx+M/3P0W4JZ4XEtEJBHcvaGxn7c3cP6N\nwI2tF5GIpLqMrKheVFQUdAhA8sQBiqU+yRIHJFcskv5S/e9bqscPqf8dUj3+lrJE9WCbmXrLRdKA\nmeFpOlA9HtTWiaSHlrR1GdlTJSIiIhJvSqpERERE4kBJlUiSq6yubPH0XhERSRwlVSJJrKqmijuX\n3MmS9Q2WRBIRkSShpEokSbk7c96ZQ7ucdozsMzLocERE5DCUVIkkqSc+eIIte7fwvcLvkWX6pyoi\nkuzUUoskoZJVJZR+UsqPR/2Y3OzcoMMREZEmUFIlkmTe2fgOC1YsYNJpk+jYpmPQ4YiISBMpqRJJ\nIh9v/5j7372fH4/8MQV5BUGHIyIizaCkSiRJbN67mT+8/gcuO/ky+nXpF3Q4IiLSTEqqRJLA7ord\n3PbqbZw/+HyG9RgWdDgiItICSqpEAlZRVcGdS+7k1L6n8oV+Xwg6HBERaSElVSIBqvEa/lz6Z3p3\n6s15x50XdDgSJzU1QUcgIkFQUiUSEHfnwfcepKqmim8N/xZmzVoMXZLY7t1BRyAiQVBSJRKQf330\nL1btWMUPR/yQ7KzsoMORONqxI+gIRCQISqpEAvDK2ld4cc2LXDXqKtrltAs6HIkzJVUimUlJlUiC\nLduyjMeWPcZVo64iv11+0OFIK1BSJZKZ4pJUmVm+mT1iZsvMLGRmp8bjuiLpZu3Otcx+azYTR0yk\nV6deQYcjrURJlUhmyonTdW4DFrj7f5pZDpAXp+uKHKK0tJS5c58GYMKEcRQWFgYcUdNsLd/KnUvu\n5JITL2Fg14FBhyOtaOfOoCMQkSDEnFSZWWfgC+5+GYC7VwG7Yr2uSH1KS0uZOHEm2dmXAbBo0Uxm\nzbom6ROr8spy7lhyB2cdexaFvZI7VomdeqpEMlM8Hv8NAMrM7F4zKzWzP5pZ+zhcV+QQc+c+TXb2\nZRQUjKWgYCzZ2Zcd7LVKVpXVldz1+l0M7T6UsceMDTocSQAlVSKZKR5JVQ5QCPze3QuBcuBncbiu\nSMpzd+57+z7y2+bz9SFfDzociWJms81sk5m9G3VspJktMbO3Ij9HRL03xcxWRMaOntXYtZVUiWSm\neIypWgesdfc3Iq8fBYrrO3HatGkH94uKiigqKorD7SWTTJgwjkWLZlJWFn5dXX0fEyZcE2xQdYRC\nIeYvnA9A7gm51HSqYdKpk1K2uGdJSQklJSVBh9Ea7gXuAOZEHbsZuMHdnzGzc4BbgC+Z2RDgG8AJ\nQF/gWTMb5O5e34WVVIlkppiTKnffZGZrzew4d18OjAWW1ndudFIl0hKFhYVMmnQ2t99+GwCTJk2I\n+3iqWAbCh0Ihrr/lenKH5rKlegub/7WZB7/7ILnZuXGNMZHq/gI0ffr04IKJI3d/0cz61Tn8CVBb\n56ILsD6yfz7wUGTM6CozWwGMAl6r79pKqkQyU7xm/10NPGBmucDHwHfjdF2RzygtLeW2254iO3sS\nALfddh8nnHDCwcQn1pmBsQ6En79wPrlDc8k+Opu92/cyKGsQzy96npEnjWxWHBKYnwEvmdlvAAM+\nHzneB3gl6rz1kWP10uw/kcwUl6TK3d8B9H8NaXXRA9UBysrCxwoLC+MyM7Cx6zfVnpo9bNm2hWE9\nhrF3994mf06SwmzgKnd/wsy+DtwDnNnciyxdOo3ajnkNdRBJDfEY6hCvniqRwMUjIYrVqWecyt0P\n3k1/+rN3z14qQ5WMnzw+YfeXmJ3q7mcCuPujZvbnyPH1wFFR5/Xl00eDh8jL+zSpEpHUEI+hDkqq\nJKW09kD1WK6/c/9Ontn5DL+68FesK10HwPjJ4xk6dGjc4pO4s8hWa4WZfdHdXzCzscCKyPEnCQ9x\nmEn4sd9AYElDF9WYKpHMZA1MXon/jcwamigj0iwNjZuq+/ivuvq+FhUGbcm4rP1V+7n15Vsp7FXI\nVwd9tVn3SzVmhrun5lTGKGb2IFAEdAM2AVOBd4G7gDbAfuBKd38rcv4U4AqgEpjk7s80cF3PznYq\nKyFFJ3yKCC1r65RUSVoJYgmb6ppqfv/67+naviuXnnhpypZOaKp0Sapai5l5hw7Oxo3QsWPQ0YhI\nSympEmlF9SVs7s6cd+aw+8Burhx5JVmWlbJrEzaVkqrGmZn36eO8+ir07Rt0NCLSUi1p6zSmSqQJ\nGppZuL7Dejbs3sC1p197MKFKxbUJJb66dAmPq1JSJZJZlFSJNEF9Mwt/88jdDDizO8Wji2mb07bB\n8xI9A1GCl5+vweoimSgea/+JZJzyjitZkb2Mq0+9mk5tOwUdjiSZ2p4qEcks6qkSaYIJE8axYMFU\n1m99iKoOuzkw5HXuGTeTHh16HHJesq9NKK1PSZVIZlJSJVKP6MHmI0YM5JU3XmF33nvUnPwe+wp2\n03/j0Rzb9dhDPldYWMisWddEDVTXeKpM1KWLlqoRyURKqkTqiB5sXl6+irsf/yldB7Zl3yn72N91\nP4X9Csnvnc/8hfPrLexZWFioRCrDqadKJDNpTJVIHdGDzQ9klZF1fH/KrYIDRxygbfu2lG8rDzpE\nSXJKqkQyk5IqkcNwHB/s2Hoja10Wu9fsDq/pd6bW9JP6afafSGbS4z+ROiZMGMfjj1/LsjVTqKos\np7LLKrp1PJK+Wcex8dWNnD3mbK74yRVa008apJ4qkcykpEqaJd2rhQO89NJLrDvwGtV9quFYwypg\n1KZRnDTkJMZfrgWS5fA0UF0kMympkibLhGrhoVCIX/7pl9ScUUNOtxyq9lTRoaYj27dtp/inxUGH\nJylCPVUimUlJlTRZJlQLn79wPjbYsL5GTacacmtyqfyoEnoc/rMitZRUiWQmDVQXqaNPvz5UVVbB\nbqjaXkXV6ira1xRQXHwjpaWlQYcnKUAD1UUyk5IqabIJE8ZRXX0fZWXPUVb2XKRa+Ligw4qrL3zx\nC+yq3sXg7MF0/rgzeW/l0d2H8cknl7F48SgmTpypxEoOqzapcg86EhFJJPM4/as3syzgDWCdu59f\nz/ser3tJcNJxoHooFGL+wvlUeiUb+25keNfhbH9vOwDL3/uEpUvPi3rk+RxjxixhxowpQYYcKDPD\n3S3oOJJVbVvXvj1s3Qp5eUFHJCIt0ZK2Lp5jqiYBS4HOcbymJJl0qxYeCoW4/pbryR6SzcdVH5Nb\nksvEH01k2NhhABQX3xhwhJKqamcAKqkSyRxxSarMrC/wVeB/gGvjcU2R1hLd27av5hNyhuSwo/sO\njvAj6JrTlX8++0+GDQsnVVogWVqqdrB6r15BRyIiiRKvnqqZwGQgP07XE2kVdctCrN1yHV0urqTD\nER0Y3nM4W7Zu+cz5WiBZWkozAEUyT8xJlZmdC2xy97fNrAho8PnjtGnTDu4XFRVRVFQU6+1FmmXu\n3KepqCjiQNYbAFivk1m/+h+M7nwaW3ZsCS8/M/mzy8+k2yPP5iopKaGkpCToMFKOZgCKZJ6YB6qb\n2f8C3wKqgPZAJ+Bxd/9OnfM0UF0Cd8UVP+Xxxa/S9qQjqWy3k70HPmSsj2bM6BEAjD9TFdMPRwPV\nG1fb1l18MVxwAXzzm0FHJCItEchAdXf/OfDzSABfBP67bkIlUiuW2YPxmHnYoWsNDF5HVbc89nVa\nTduPazj22F6qli5xp6VqRDKPKqpLwsSyzE08lsgJhUKsWLmCzj2y2V3wHkdZT3qdcDQc+HSWX7qU\niZDgaUyVSOaJa/FPd3+hvhpVIvDZZW4KCsaSnX3ZwZ6n1vwsfFo6YW+/vZTVlOGfVNE3qydVS6tY\n9NRKFi8epeKeGcbMZpvZJjN7N+rYQ2ZWGtlWmllp1HtTzGyFmS0zs7MOd30lVSKZRz1VkhHmL5xP\n1pAs9h+xn5N3n8yOF3fQZlcb+nc9kaUdz0vr9QylQfcCdwBzag+4+8W1+2Z2K7Ajsn8C8A3gBKAv\n8KyZDWpsoGh+Pqxa1TqBi0hyUlIlCdPSmk+lpaVs3LieVav+lz179tKxY4dm14uq8RpWV62mW243\nBg0cxObqzZyeezrbPqlp6deRFOfuL5pZv0ZO+QZQFNm/AHjI3auAVWa2AhgFvNbQh9VTJZJ5lFRJ\nwrSk5lP0WKru3YexZctNfO5zI7n66qaPp3J3bLBR/VQ1nXM6s3nL5oOlEyoqKlTcUw5hZl8ANrr7\nx5FDfYBXok5ZHznWICVVIplHSZUkVHNrPtUdS9WhwyB69lzSrGv866N/UdWxivu/dz8Ln18IwPjJ\nn5ZOUHFPqcc3gb+19MPTpk1j7Vp4/30oKVFNPpFUEI+afEqqJK29vPZlXlzzIsWji8lvl0/h8EMT\npkwv7imfZWbZwNeA6L8U64Gjol73jRyr17Rp01i2DF5+GZRPiaSGukXJp0+f3uxrxHX2n0i8TZgw\njurq+ygre46ysucij+fGNemzS7cs5fFlj3PVqKvIb6cVlKRexqGrQJwJLHP3DVHHngQuNrM2ZjYA\nGAgsaezCevwnknlirqje5Buporq0UHOKfoZCIeYvnM+Omh18cuQn3HD2DQzsOjBRoWaEdKmobmYP\nEh6I3g3YBEx193vN7F7gFXf/Y53zpwBXAJXAJHd/poHrurtTXg7dusG+fa36NUSklbSkrVNSJWmj\nthaVD3FWVK6g5+qe3D3pbi07E2fpklS1ltq2zh3atoXdu8M/RSS1tKSt0+M/SRvzF87Hhhib8jcx\nuP9gCgYVMH/h/KDDkgxlpqVqRDKNkipJG9VezaqqVXRt35U+nRud7S6SEBpXJZJZlFRJWnB3Dgw8\ngH1i5G3MY9MHm8K1qM4cH3RoksGUVIlkFpVUkLTw6NJH6ditI/f/4H6efi48qD26FpVIEPLzlVSJ\nZBIlVWmqOTPmUt2zHz9LaEuI60dfT15uHiefeHLQIYkA6qkSyTRKqtJQ9NIuAIsWzWTWrPSpFD5v\n3jxm3T8LgK9c9BU2F2ym+Ixi8nLzAo5M5LOUVIlkFiVVaSh6aReAsrLwsaYkVcnewzVv3jy+P+37\nZI/Kpiq7ihf++QK3X3A7Xdt3DTo0kUNo9p9IZtFAdTmotodr8eJRLF48iokTZ1JaWhp0WAeFQiEm\n/2oy+zrtI7d7LjX9a+iQ34HH5j4WdGgi9VJPlUhmUU9VGpowYRyLFs2krCz8Ory0yzWH/VwsPVyt\nrbaw544BO6hoV8HGDRspsAK8SgVlJXl16QIbNhz+PBFJD0qq0lBhYSGzZl0T9RgvtcZT1X0E2bZt\nW66beh2ru6/m2MHH8samN6Acdr+wm/a72zNx2sSAIxapn2b/iWQWJVVpqrCwsNmJVEt7uOKp7iD7\nBQumkj9wO9ttO9tqtvHJrk8Y0H0A29/ezhHlR3DLtFu44IILEhqjSFPp8Z9IZol5TJWZ9TWz580s\nZGbvmdnV8QhMEq+2h2vMmCWMGbMkkBmD0Y8gCwrGsrPiSDblb+OEL59A5b5KqvdXk70um1EdRvH3\nv/xdCZUkNQ1UF8ks8eipqgKudfe3zawj8KaZPePuH8Th2pJgLenhSoTd+bvpfWJvql6tor/15+bp\nN6uwpyQ99VSJZJaYkyp33whsjOzvMbNlQB9ASZU0W+0jyOXLX2LTnvlUVa6kYFsHVq1ZxcDcgdAJ\nbp6shEpSg5IqkcwS1zFVZtYfOBl4LZ7XleAlqn5VYWEhRUW9mPnwr7FCo21BGzbu2815a89jxPEj\ntPSMpBQNVBfJLHFLqiKP/h4FJrn7nvrOmTZt2sH9oqIiioqK4nV7aUWJrNAeCoW49x/3kjXayOmf\nw/7K/XTa2ok9ZXso/mlx3O8nh1dSUkJJSUnQYaSkjh1h/36orITc3KCjEZHWFpekysxyCCdU97v7\nvIbOi06qJHXEu35VY71e8xfOx3oYWR2zqG5fTa7lUrmrMubvIC1X9xeg6dOnBxdMijEL91bt3AkF\nBUFHIyKtLV4V1e8Blrr7bXG6nqSpplRtP3rI0VTurcQ/capXVuPvOBO/rVpUkpo0A1Akc8SjpMJo\n4FLgy2b2lpmVmtnZsYcmyWLChHFUV99HWdlzlJU9F6lfNa5F16pbMiE7+7KDvVYA48aOY0+bPRzT\n7Rjy38+nc2lnbrzqRpVOkJSlweoimSMes/9eArLjEIskqURVaK/xGl7e/zKXnnkpuStysQHG+DOD\nGZie7AtLS+pQUiWSOVRRXZokXvWrGqra7u7MfX8uFVUVTPnqFHKygvurmciB+ZL+NANQJHMoqZKE\naqjX6+mPnuajbR9x3eevCzShguReWFpSj3qqRDKHkipJuLq9XkvWL6FkVQnFZxTTPrd9gJGJxJ8G\nqotkDiVVknChUIj5C+cDMOTUIbyw+wWuPf1aurTrEnBkYcmwsLS0PjObDYwHNrn78KjjVwFXEl6C\n65/u/rPI8SnA5ZHjk9z9mabcRz1VIplDSZUk1Lx58yj+bTFZQ7Lo2r0rdz9yN3d84w56d+oddGgH\nJWpgvgTuXuAOYE7tATMrAs4DTnT3KjMriBw/AfgGcALQF3jWzAa5ux/uJl26wEcftUL0IpJ0lFRJ\nwoRCIYpvLmbnsJ3k9MlhXfk6BncczPuvvs+5p50bdHifkawLS0v8uPuLZtavzuEfATe5e1XknEh/\nJRcAD0WOrzKzFcAomrAklwaqi2SOeBX/FDms+Qvnk90zm9yOuexvu5/27duzd8veoMMSiXYcMMbM\nXjWzRWb2ucjxPsDaqPPWR44dVrdusGVLnKMUkaSknipJqO4DurN++3pyc3KxHUbNBzWM/8n4oMMS\nqZUDHOHup5nZSOAR4JjmXiR6Sa5jjy3iww+L4hWfiLSSeKxzak0YEhAXZtaU4QeSxt5//30umXUJ\nFR0ryF6eTc2mGmZcP0PV0lOMmeHuFnQc8RB5/PeP2oHqZrYAmOHuL0RerwBOA74P4O43RY4/BUx1\n90Me/9Vt66qqoFOncGmODh1a+xuJSLy0pK1TT5XEVWOVyJdnL+fML59Jt9XdyD4yO7Bq6SJRLLLV\negL4MvCCmR0HtHH3rWb2JPCAmf2W8GO/gcCSptwgJwcGDYIPPwQN0xNJb0qqJG4aq0ResqqEtze+\nzS/H/5IObfTrugTPzB4EioBuZrYGmEp4cfh7zew9oAL4DoC7LzWzh4GlQCVwZXO63ocMgaVLlVSJ\npDslVRI3DVUit17GghULuH709UqoJGm4+yUNvPXtBs6/EbixJfeqTapEJL0pqZK42707xKad89m7\n9yM+3l7FX98t4+pTr6YgryDo0EQCMWQI/PWvQUchIq1NSZV8RmNjog5nwoRxPP74tawr/xh6GNlH\nl/Nybhsu7fR7+nWpWw5IJHOop0okMyipSgOxJEJ1r9PQmKimaNu2LZVdV5E1YjtZedk4VfTscBQf\nLvkQPt+ikETSwsCBsGYN7N8P7doFHY2ItBYV/0xxtYnQ4sWjWLx4FBMnzqS0tLRF14oeE1VQMJbs\n7MsOJmtNMX/hfPIK8ygY2pV2x+eSV5BHxUcVLYpFpKnM7FgzaxvZLzKzq80sORaSjGjTBo45BpYv\nDzoSEWlNSqpSXKyJULwd2f1Idu3Zhe93snZkUb2pmvFnqrintKrHgGozGwj8ETgKeDDYkA41dCiE\nQkFHISKtSUmVAOEer02bNrJq1f9j9epZlJU9R3X1fUyYMK7J1zj3K+eybes2urftTvcN3enyQRdm\nXD9DtaiktdVE1uT7D+AOd58M9Ao4pkNoXJVI+tOYqhQ3YcI4Fi2aSVlk2ddwInRNkz5bOxZr48b1\nvPnmBjp1+jEFBWPZsuU2TjnlBCZNavp4KoDVbVYz5ktj6Lm2JzlH5DD+JyruKQlRaWbfBP4LOC9y\nLDfAeOo1ZAg8/HDQUYhIa4pLUmVmZwO/I9zzNdvdZ8TjunJ4hYWFzJp1TdRA9aYlQtGD0levfphd\nu86isHAE/fvn07FjB448cslhrzNv3jxm3T8LgKKvFbGz505+fd6v6dy2c8zfS6QZvgtMBP7H3Vea\n2QDg/oBjOoR6qkTSX8xJlZllAXcCY4ENwOtmNs/dP4j12tI0hYWFzZ7xFz0Wa8uWJeza1Z0NGzaT\nn5/fpM/fcccdTLlzCjbSyOmQwwsLXuCuC+9SQiUJ5+5LgasBzOwIoFMy/mJ33HGwciUcOBAeuC4i\n6SceY6pGASvcfbW7VwIPAVohN4X07j0OmEN5+ctNGksVCoX45R2/pHJEJTWDatjTcw/t8tox96G5\niQtaJMLMSsyss5l1BUqBP0XW6EsqbdtCv36wYkXQkYhIa4nH478+wNqo1+sIJ1qSxOqOxerXr5KR\nI0vp2XNDo48QQ6EQ1029jvKqctgH1dnV5JJLZXnlwXPiVTdLpIny3X2XmX0PmOPuU83s3aCDqk/t\nI0ANNRRJTwkdqD5t2rSD+0VFRRQVFSXy9hLl0LFYvz5s8hMKhbj+lutZ3X01uaflUr6vHHs/iyw3\ncktzmXjrxJgLiEryKSkpoaSkJOgwGpNjZr2AbwC/CDqYxmhclUh6i0dStR44Oup138ixQ0QnVRK8\n5ozFqu2hWt19NX2G9GHt2rVkfZKNP5uLVbWle/tBHHXUUQ0uqqykKnXV/QVo+vTpwQVTv18CTwMv\nufvrZnYMkJQP2YYMgSefDDoKEWkt8UiqXgcGmlk/4BPgYuCbcbiuJInaHqo1NWvYVrONjes30tbb\nU7m5K53bDWFIv1upqNgYaNFRyVzu/gjwSNTrj4GLgouoYUOGwE03BR2FiLSWmAequ3s18BPgGSAE\nPOTuy2K9riSP+Qvnkzs0lyFjh1C1p4qqA1XUfOS0/7gXQ/rdSqdOnw4QmTBhHNXV91FW9lyLCoiK\nNJeZ9TWzv5vZ5sj2mJn1DTqu+gweDB99BFVVQUciIq0hLmOq3P0pYHA8riXJq7xLOT1O6oG/5nTd\n25Vd7bpSUbGRioqNB4uOtrRulkgM7iW8LM1/Rl5/K3LszMAiakBeHvTuHU6sjj8+6GhEJN7M3RNz\nIzNP1L0kvkKhED+47Qds7reZQbmDsKXGzZNvpqKiotVm+WkGYfIyM9zdgo6jlpm97e4nH+5YAuNp\ntK077zz47nfha19LYFAi0mwtaeu0TI3UKxQKMX/hfABOPO1EBhUNYszGMXTJ6sL4yZ8uP9MayY5m\nEEozbTWzbwF/i7z+JrA1wHgaVTsDUEmVSPpRUiWHqB2Ynjs0l/2+n7sfvpvf/efvOP+S8xNyf80g\nlGa6HLgDmAk48DJwWZABNWbIEHhaczpE0lI8KqpLGqktnbBq1yq8nbOp8yb69ujLstdSf+5BaWkp\nxcU3Ulx8I6WlpUGHI3ESWc3hfHfv7u493P1CknT2H4QLf6pWlUh6UlIlB4VCIa6ceiVv2puszlnN\n8288T4d9Heia3TWhcbTGDMLaR4qLF49i8eJRTJw4U4lVers26AAacvzxsHw5VFcHHYmIxJse/6WZ\nWAZ4z/7rbD7I/YCc43Ko2F+Bb3I2LNhAfp98xk8e31ohH6I1ZhDqkWLGSZqB9HV17Ag9eoQXVx44\nMOhoRCSelFSlkVgGeNcOTN87YC/t27Qnv30++9fvJ2d7Djf/7uaDA9MTpTnV3kXqkdRTjWsHqyup\nEkkvevyXRqJ7YwoKxpKdfVmTqpzXDkyvPraaivIKdqzdQe6aXDqs6cD4seMbTKhSaYySipKmHzPb\nbWa76tl2A72b8PnZZrYpevFlM5tqZuvMrDSynR313hQzW2Fmy8zsrFhiHzIEQqFYriAiyUhJVYar\nHZi+sv1Kup3WjbaD2tL2w7ZUv1bN4CMGc8V3rqj3c6k2Rqn2keKYMUsYM2aJSjSkAXfv5O6d69k6\nuXtTeuHvBerLrH/r7oWR7SkAMzuB8ILNJwDnAHeZWYsfMZ5+Oixa1NJPi0iy0uO/NDJhwjgWLZpJ\nWVn4dW2V84bMmzeP4puL2bZ/G5VHVFK9pZoRx4xgc8Vm+m3px63Tb22wlyoVxyjpkaJEc/cXI2uW\n1lVfsnQB4SW4qoBVZrYCGAW81pJ7jxsXLgC6Ywd06dKSK4hIMlJPVRppTm/MvHnz+N7/+x7re6+n\nakgVu/fuJmdTDpuXbWbAvgGNJlQiae4nZva2mf3ZzPIjx/oAa6POWR851iIdO0JRESxYEEOUIpJ0\n1FOVhGLoBo+PAAAgAElEQVSZwdeU3phQKETxzcXsH76fqt5VVFBBp02dyHk7h34D+nHz9PASNMXF\nNzYYQ3N7xURSxF3AL93dzezXwG+A7zX3ItOmTTu4X1RURFFR0SHnXHghPPEEXHJJi2MVkTgqKSmh\npKQkpmto7b8kU3cGX3X1fXEf/zPjdzOYs3gO2/puY2uXrdgBo82/29BnQx8e++NjVFRUNCkGrc+X\nmZJt7b9YRB7//cPdhzf2npn9DHB3nxF57ylgqrsf8vivqW3dli0waBBs3Ajt2sX8VUQkzrT2XxpI\n1Fil7gO6s37HejrmdqRyUyXtPmzHjF/NYOjQoRQX39ikGDRGSdKAETWGysyOdPeNkZdfA96P7D8J\nPGBmMwk/9hsILInlxt27w/Dh8NxzcO65sVxJRJKFkqoMUluLauMnG1m/ez29uvfClhs1m2qY8asZ\nXHDBBUGHKJIwZvYgUAR0M7M1wFTgS2Z2MlADrAJ+CODuS83sYWApUAlcGY+u99pHgEqqRNKDHv8l\nmdZ6/FdbiypnSA7rq9ez/ePtnFdwHjVVNezdlkVBQc+Dj/AS8QhSUlc6Pf5rDc1p6z7+OFxeYcMG\nyM5u5cBEpFla0tYpqUpC9Y1VinX80ozfzeCVylc40PsAW/Zu4cgdRzJgywBefWF7vcmTxktJQ5RU\nNa65bd1JJ8Fdd8Ho0a0YlIg0m5KqFNZYEhNLz1HtI7+SF0vYfMxmao6p4aSeJ7Hjox1sX7ifA7sn\nR42deo4xY5YwY8aUuH8/SR9KqhrX3LZu6lQoL4dbbmnFoESk2VrS1qlOVRI4XHXyli4/M2/ePC66\n8iL+8uFfWD9gPcs2LCPvozx2fLSDylAlA486vvW+lIg0yYUXwt//DvqdUyT1xTRQ3cxuBs4DKoB/\nA991913xCCyTtMaMv9paVDuH7SS7Tza7yncxsNdAOq/szOl9T2f85PEHSyeo1pRIcE4+GaqqwmsB\nDhsWdDQiEotYZ/89A/zM3WvM7CZgSmSTOGpJoc3Zc2azbf82KlZXYD2NDnkdqFpfRdEZRRT/tPjg\nebNmXRP12FGD0UUSzezTWYBKqkRSW9zGVJnZhcBF7v7tBt7XmKoGNGXMVHMGjodCIS668iLKBpSx\ns/1ObLXRuUNnCjYX8Nhdj2n5GYmJxlQ1riVtXUkJXHcdvPFG68QkIs0X6EB1M3uS8IKjDzbwvpKq\nRsRjtl0oFGL2nNnMf24+5YPL2dV7FzntcjjwwQHav9OeP9/yZ9WikpgpqWpcS9q6qio48kh46y04\n6qhWCkxEmqVVKqqb2UKgZ/QhwIFfuPs/Iuf8AqhsKKGq1ZT1sDJVrNXJQ6EQP/r5j/hw+4fszdvL\n/o77aVfTjj7eh+qCas6+6OyUSqhU0iF5xGM9LGlcTg6cfz787W9w/fVBRyMiLRVzT5WZXQZ8H/iy\nu1c0cp56qlpJKBTiuqnX8ebKN8kZlkP10dVs+3gbba0tR3c6mgH7BnDz5JtT5rGfio8mN/VUNa6l\nbd2bb4bHVn38MeTmtkJgItIsCS+pYGZnA5OB8xtLqKT11FZKX919NfuO2UfZ5jIO2AG6detG3to8\n+m3pl1IJFbS8hIRIKvvc58ILLM+dG3QkItJSsc7+uwNoAyw0M4BX3f3KmKOSJpv919msbL+SnG45\n5OTkUF5VTuVLlbTp0YYTup/ArdNvrTeh0uM1keRz3XXw85/DpZeGZwWKSGqJqafK3Qe5ez93L4xs\nSqgSKBQKsWDxArbXbGdru60cyDtAj/Ie9N7Zm0v7X8pd0+9qMKFqrNho0CZMGEd19X2UlT1HWdlz\nkRIS44IOS6TVnX02VFbCc88FHYmItESsPVUSkHnz5jH5V5MpKy/DPjCqulXRfk978lfl89hfGi+b\n0BrFRuOpsLBQ9bMkI2VlwX//N9x6K3zlK0FHIyLNpaQqBc2bN4/vT/s++4buo8IqqC6vpteKXuRX\n5zPutHEpNX6qIbHOhhRJVZdeCjfcAO++C8OHBx2NiDSH1v5LQbPun0X2qGy6n9Id+kNWtywOfHyA\n/p37c8V3rjjs5/V4TSR5tW0LV10Fv/1t0JGISHPFrfjnYW+kkgpxc87Xz+Gtbm/hgxyvdsrfK6f3\nB735+1/+3uReKg1Ul5ZSSYXGxaOt27YNBg6E996DPn3iFJiINEugFdUPeyMlVXHzxBNP8J17vgMD\nIW9vHjVLavjTtD+lVHFPSV1KqhoXr7Zu0iRo1w5mzIhDUCLSbEqq0lgoFGL+wvkAdBrWiZf//TJb\nnt1Clmcx8dsTlVBJwiipaly82rqVK2HEiPDPzp3jEJiINIuSqjQ1b948in9bTNaQLPK657Fj9w4e\n+K8HOPXkU4MOTTKQkqrGxbOt+/a3YcAA+OUv43I5EWmGhFdUl9YXCoUovrmYncfvZEefHSyvWk6v\nTr20FptIBvjf/4Xf/x5Wrw46EhFpCiVVSW7+wvlk98wmu1M2+3P30ymvE1u3bA06LBFJgKOOCs8E\nLC4OOhIRaQolVSmg6zFd2bV7F23L2lKztoaapTWMP3N80GGJSAJcfz28/DK8+GLQkYjI4SipSnJf\n/tKX2Vi1kWPzj6Xrv7uS/34+M66dkRYFPkXk8PLy4Kab4Kc/hZqaoKMRkcZooHoSqp3pV+3VbO23\nlaM6HEXFsgoAxp85nqFDh6rOlAQmXQaqm9lsYDywyd2H13nvv4FbgAJ33xY5NgW4HKgCJrn7Mw1c\nN+5tnTuMHg3f/z5897txvbSINECz/9JAKBTi+luuJ2dIDqurVlP9STV/++HfGDZs2MFzSktL+c53\nfs22beEyCl27zmPOnBuUWElCpFFSdQawB5gTnVSZWV/gz8Bg4HPuvs3MTgAeBEYCfYFngUH1NWqt\n1da9/jpccAF8+CF06hT3y4tIHZr9l+JCoRDXTb2Ole1XsrnjZtr3aM+Aowbwz2f/+Znzbr/9L6xa\ndRb79n2Vffu+yqpVZ3H77X9JaKylpaUUF99IcfGNlJaWJvTeIvHg7i8C2+t5ayYwuc6xC4CH3L3K\n3VcBK4BRrRvhZ40cCWeeGZ4RKCLJSUlVkpg3bx4X/eAi3lz5Jut9PUvXLqVXTi+y7NA/omXLVgLd\nadMmvEH3yLHEKC0tZeLEmSxePIrFi0cxceJMJVaSFszsfGCtu79X560+wNqo1+sjxxLqxhvhT3+C\nDz5I9J1FpClygg5AIrWoflvMzmE7qampYe+OveRX5PPBjg8YsG8A4yd/dqbf8ccfTSg0hwMHukSO\nzOH4449OWLxz5z5NdvZlFBSMBaCsLHxMjx8llZlZe+DnwJmxXmvatGkH94uKiigqKor1kgD07g2/\n+lW4KOjLL0NublwuKyJASUlJzDUglVQlgfkL55M1JIusPll4G6fbB93wd5x+A/px8/SbD5npN2nS\n5bzxxg1s3/4wAL17VzJp0uVBhC6STo4F+gPvmJkRHjtVamajCPdMRf/m0jdyrF7RSVW8TZwI8+aF\nHwNOndpqtxHJOHV/AZo+fXqzr6GkKkl07d6VdeXr6EQnanJryG+Xz63Tb623dEJhYSH33//rqNl/\nP0xoL9GECeNYtGgmZWXh19XV9zFhwjUJu79IHFlkw93fB448+IbZSqDQ3beb2ZPAA2b2W8KP/QYC\nSwKIFzO45x445RT46lfDY61EJDlo9l8SeOWtV/j2nG/TpVMXyreUU7O0hhnXzkjqRZJV0iFzpdHs\nvweBIqAbsAmY6u73Rr3/MTCiTkmFK4BKElxSoT5z54Z7qkpLw7WsRCS+AiupUF9Nl3rOUVJVj/1V\n+7nlpVvoXtmdne/vBD6tRSWSjNIlqWotiWzrLrkECgrg9tsTcjuRjBJIUlVfTZcGzlNSVUdVTRV3\nLrmT7nndueTESwgP4xBJbkqqGpfItm7bNjjpJLj3XvjKVxJyS5GMEVSdqvpqushhuDv3v3M/uVm5\nfPPEbyqhEpFm69oVZs8OV1nftCnoaEQkpoHq0TVdlBQ0z5MfPsnGPRu59vRrD9ai0jglEWmus86C\nyy6Diy6C55+HNm2Cjkgkcx02qTKzhUDP6EOAAzdwaE2XRjOr1qrdkmoWr17MGxve4PrR19M2py3w\naUHN7OzLAFi0aCazZl0Tc2KlRE1iFY/aLdK6pk+H996Dn/wE7r47PENQRBKvxWOqzGwY4fWvygkn\nU7V1W0a5++Z6zs/YMVW1CyQDDBo5iFf2vsLk0ZPp0aHHwXOKi29k8eJRUQU1n2PMmCXMmDGlxfet\nm6hVV98Xl0RNMpvGVDUuqLZu9244/XS48srwJiKxaUlb1+LHf43VdGnpNdNR7QLJuUNzKa8p5+7H\n7uYPF//hMwlVa1Hlc5HM0alTuCjo6NEwZAhk6IMAkUDFc+0/5zCP/zLR7DmzWbVrFWs+XsNqW02/\nnv14++W3DzlvwoRxVFffR1nZc5SVPRcpqDkO0OLFItI0xx4LDzwAF18MKxO3HKiIRMStorq7HxOv\na6WLUCjEglcXsP2E7VQcUUGbN9rQ56g+cNSh5xYWFjJr1jVR45/Cj+liGWulyucimWfsWPjFL2D8\neFi8GLp1CzoikcyhZWpa0fyF8+n1xV5srNxIm/ZtyD4imw2vbmD85ePrPb+wsPCQZCmWR3gNJWoi\nkt5+8hPYsCE8M/C556BLl8N/RkRip6SqFdV4DdvabWNQz0HYTmNP1h7OHnN2Qqul15eoiUh6Mwsv\nuFxeDuecA888Ex5zJSKtK55jqiSKu5N9fDaVGyvptbcXvXN7M2DfAK741hXNuk5jY61ERBpiBr/7\nHZx4Ipx/fjjBEpHWpQWVW8lTHz3F6+tf59wjzmXh8wuBlq/pp1pTkkxUUqFxydbWVVeHi4Nu2RKe\nHdi2bdARiaSGwBZUbtKNkqyhaU2vrXuNJz54guIziunSToMZJL0oqWpcMrZ1VVXwzW/C3r3wyCPQ\noUPQEYkkPyVVSeCDsg/4c+mfufb0a+ndqTeQuJ4m9WhJIiipalyytnWVlfDDH4Yrr//zn9Cj9Uvl\niaQ0JVUBW7drHb979Xf84HM/4LhuxwGJq2qu6umSKEqqGpfMbZ17eEmbv/4V/vUvGDQo6IhEkldL\n2joNVI+T7fu2c+eSO7l42MUHEyqA2267h3Xr8tiyZQm5uUeQnX3Zwd6keIouvVBQMLbV7iMiqcsM\npk2DKVNgzBh49dWgIxJJL0qq4qC8spzbX7udsQPGMqL3iIPHS0tLeeaZZezadRZbt47i3Xdnsnfv\nigAjFRGBK66Ae+4Jzwp85JGgoxFJH0qqYlRVU8UfXv8Dxxccz1eO+cpn3ps792kKCiaRk3MMMJyq\nqgvZsmVOq5REUOkFEWmOc86Bp56C4mKYNAkqKoKOSCT1KamKgbtz39v30aFNB/5z6H9iduij144d\nOzB8+DF067aDzp33MG7cyFYZ51RbPX3MmCWMGbNE46lE5LAKC+HNN2HNGjjjDK0XKBIrDVSPwWNL\nH+Pf2//NNaddQ2527iHvBz14vO5sQECzAyVmGqjeuFRs69zhttvCVdjvvhv+4z+CjkgkeJr9l0CL\nVi5i0apFFI8upkObhou+BFXmoG5Ct2vXbzBrR6dOPwY0O1BaTklV41K5rXvtNbj4YjjzTLj5Zq0Z\nKJmtJW2d1v5rgbc+eYunPnqKyaMnN5pQQXBr79VdiHn16oeBzzNgQPMXZhaRzHDqqfD22/Czn8HQ\noXDHHfC1rwUdlUjq0JiqZvr3tn/z13f/ypUjr6QgryDocERE4io/H/7wB3joIfj5z8NJ1YYNQUcl\nkhqUVDXDpj2bmPXGLC4/5XL6dekXdDiNqjsb8Igj1tK16zzNDhSJMLPZZrbJzN6NOvZLM3vHzN42\ns2fNrG/Ue1PMbIWZLTOzs4KJOnG+8IVwr9WwYXDSSfCb38D+/UFHJZLcNKaqiXZV7GLGizP46qCv\nMvro0UGH0yQaqC6tIV3GVJnZGcAeYI67D48c6+jueyL7VwHD3f37ZjYEeAAYCfQFngUG1deopXpb\nV59ly8KPBN95B379a7jkEsjSr+SS5jRQvZVUVFXwm1d+w/Cewxl/3PigwxEJVLokVQBm1g/4R21S\nVee9nwFd3P1nkX139xmR9/4FTHP31+r5XMq2dYfzf/8HkyeHa1rNmBEe0F5PJRmRtBDIMjVmdlWk\nO/w9M7sp1uslm+qaau5+8276du7LuYPODTocEWllZvZrM1sDXAbcGDncB1gbddr6yLGM8oUvwCuv\nwA03wFVXwWmnweOPQ3V10JGJJIeYZv+ZWRFwHnCiu1eZWVqN3HZ3HnjvAQzj0hMvrbe4p4ikF3e/\nAbjBzIqB3wHfbe41pk2bdnC/qKiIoqKieIUXODO46CK48EKYNy/cYzVlClx3HXz729CuXdARirRM\nSUkJJSUlMV0jpsd/ZjYXuNvdn2/CuSnXJT5/+Xze2fgO133+OtrmtA06HJGkkEGP/44CFrj7ifU8\n/nsKmJppj//q4w4vvBCua1VaCt/9Lnzve3DssUFHJhKbIB7/HQeMMbNXzWyRmY047CdSxEtrXuKV\nta9w1alXKaESSV8W2cIvzAZGvXch8HZk/0ngYjNrY2YDgIHAkoRFmcTMoKgIFiyARYvgwAE4/XQY\nOzZclkFrCkomOWxPlZktBHpGHwIcuAH4H+B5d59kZiOBue5+TAPXSZnf3kKbQ9z39n1c9/nr6Nmx\n5+E/IJJB0qWnysweBIqAbsAmYCpwLjAYqAI+Bn7k7psj508BrgAqgUnu/kwD102Ztq61VFSEHw3+\n6U/h3qsLLoBvfCOcaOUeuqKXSFJK+Ow/M1sAzHD3FyKvPwJOdfet9ZzrU6dOPfg6WccZrNm5htte\nvY0rR17JsV3Vfy1Sd5zB9OnT0yKpai1Kqj5r3Tp49FF4+GFYvjw8FutrX4MvfQnatw86OpGGBZFU\n/QDo4+5Tzew4YKG711sVMxUamq3lW7n5pZu5eNjFnNLrlKDDEUlK6dJT1VpSoa0Lypo14QRr3jx4\n6y0YPRrOOSe8DRoUdHQinxVEUpUL3AOcDFQA/13ba1XPuUnd0Ow9sJcZL83gS/2/xJcGfCnocESS\nlpKqxiV7W5csduyAZ5+Ff/0rvLVpA2PGwBe/GP45cKBqYEmwVPyzhSqrK5n56kyOPeJYLhpyUdDh\niCQ1JVWNS+a2Llm5w4cfwuLF4ZmEL7wQrn11+ukwcmR4GzECunQJOlLJJEqqWqDGa/jjm38kJyuH\nK065QrWoRA5DSVXjkrWtSyXusGoVvPYavP56eHvrLejdO7wO4YknfroNGKAlc6R1KKlqJnfn4dDD\nrNu1jkmnTSInK6ZaqCIZQUlV45KxrUsH1dXhNQjffRfee+/TbevW8HiswYPhuOPC26BB4WSre3c9\nQpSWU1LVTAv/vZCX177M5NGTycvNCzqcFqm7aLIWSZbWpqSqccnY1qWzXbvCswqXLw8/QqzdX7ky\nXDOrf/9wgtW/P/TtC336hH/27Rvu+cpLzaZfEkBJVTO8vv51Hlv2GMWjizmi/RFBh9MipaWlTJw4\nk+zsywCorr6PWbOuUWIlrUpJVeOSra3LZDt3hh8jrlwJq1fD+vXhEg+124YN4QHyPXvCkUeGtx49\noKDgs1vXrnDEEeEtP1+PGzNFS9q6jHzetXzrcuaG5vLT036asgkVwNy5T5OdfRkFBWMBKCsLH1NS\nJSISToBOOim81cc93NO1ceOn25Yt4bb0ww/hpZfCr7dtg+3bw9uePdC5c/jatVvt644doVOnT392\n6PDplpf36c/27Q/dcnP1qDIdZFxStWH3Bv745h/5XuH36Nu5b9DhiIhIQMw+TYwGD27aZ6qqwj1g\nO3eGE7Lo/T17YPfu8M8NG8L75eWwd+9nf+7bd+hWXQ1t24a3du0+3W/T5tOfbdqEk6/an/VtOTmf\n/ozesrPDW/R+3S0rq+GfdffNDt2veyz6eFO32vNr/3yit9pj0e/Vtx/9s6nHOnSIz+zSjEqqamf6\nfX3I1zm+4Pigw4nZhAnjWLRoJmVl4dfV1fcxYcI1wQYlIpLGcnKgW7fwFk81NeHlffbvD28VFeEx\nYQcOhPcrKqCyMrzVHq99XbtVVX36M3qr/Ux1dXirqvp0v3arqWn4Z91j7uH92p8N7bsfun+4raYm\n/N+j7vHaY9Hv1bcf/bM5xy69NLwoeKwybkzVropddG7bOegw4kYD1SXRNKaqccnS1olIbDRQXURa\nnZKqxqmtE0kPLWnrNIdBREREJA6UVImIiIjEgZIqERERkThQUiUiIiISB0qqREREROJASZWIiIhI\nHCipEhEREYkDJVUiIiIicaCkSkRERCQOlFSJiIiIxEFMSZWZjTSzJWb2VuTniHgFJiLSmsxstplt\nMrN3o47dbGbLzOxtM3vMzDpHvTfFzFZE3j8rmKhFJJnF2lN1M3CDu58CTAVuiT2k1ldSUhJ0CEDy\nxAGKpT7JEgckVyxp5F5gXJ1jzwBD3f1kYAUwBcDMhgDfAE4AzgHuMrO0Xf8w1f++pXr8kPrfIdXj\nb6lYk6pPgPzIfhdgfYzXS4hk+cNOljhAsdQnWeKA5IolXbj7i8D2OseedfeayMtXgb6R/fOBh9y9\nyt1XEU64RiUq1kRL9b9vqR4/pP53SPX4Wyonxs//DHjJzH4DGPD52EMSEUkKlwN/i+z3AV6Jem99\n5JiIyEGHTarMbCHQM/oQ4MANwFXAVe7+hJl9HbgHOLM1AhURSRQz+wVQ6e5/O+zJIiIR5u4t/7DZ\nLnePHsi5093zGzi35TcSkaTi7mkxnsjM+gH/cPfhUccuA74PfNndKyLHfga4u8+IvH4KmOrur9Vz\nTbV1ImmiuW1drI//VpjZF939BTMbCyyPV2AiIglgkS38wuxsYDIwpjahingSeMDMZhJ+7DcQWFLf\nBdXWiWSuWJOqHwK/N7M2wH7gB7GHJCLS+szsQaAI6GZmawjPYP450AZYGJnc96q7X+nuS83sYWAp\nUAlc6bF084tIWorp8Z+IiIiIhCW8orqZXRUpnveemd2U6PvXieW/zazGzLoGGEODxQYTdP+zzewD\nM1tuZsWJvHedOPqa2fNmFor83bg6qFgi8WSZWamZPRlwHPlm9kjk70jIzE4NMJYpkRjeNbMHIj3U\nEiVZ/j01RwNFUI8ws2fM7EMze9rM6h0rmwwaajtS5TuYWVszey1SRDtkZv8bOZ4S8deq22amYPyr\nzOyd2mLmkWPN/g4JTarMrAg4DzjR3U8Ebk3k/evE0pfwTMXVQcUQUW+xwUQwsyzgTsIFEIcC3zSz\n4xN1/zqqgGvdfShwOvDjAGMBmET4UU/QbgMWuPsJwEnAsiCCiAzo/j5wSmRQdw5wcRCxJKsk+/fU\nHPUVQf0Z8Ky7DwaeJ4HtUgs01HakxHeIjN37UqSI9nDgy2Y2mhSJP0rdNjPV4q8Bitz9FHevrUHX\n7O+Q6J6qHwE3uXsVgLuXJfj+0WYSHpAaqEaKDSbCKGCFu69290rgIeCCBN7/IHff6O5vR/b3EE4e\nAqkDFEm4vwr8OYj7R8XRGfiCu98LECk8uSugcHYBB4AOZpYD5AEbAoolWSXNv6fmqK8IKuG4/xLZ\n/wtwYUKDaoYG2o6+pNZ3KI/stiX8/+XtpFD8DbSZKRN/hHFoTtTs75DopOo4YIyZvWpmiyygtQLN\n7Hxgrbu/F8T9G3E58K8E3q8PsDbq9TqSoKChmfUHTgYOma6eILUJd9ADDgcAZWZ2b6Rb/Y9m1j6I\nQNx9O/AbYA3hwpc73P3ZIGJJYkn576mFerj7JggnLUCPgONpkqi241WgZ6p8h8ijs7eAjUCJuy8l\nheKn/jYzleKHcOwLzex1M/te5Fizv0Oss/8OYY0XC80BjnD308xsJPAwcEy8Y2hCHD/ns0VKW3UK\ndCOx/MLd/xE5p7bY4IOtGUuyM7OOwKPApMhvnYm+/7nAJnd/O/K4Osjp8TlAIfBjd3/DzH5HuDt6\naqIDMbNjgGuAfsBO4FEzuyTT/75mkKB/wTisum2HHVovLGm/Q+RpxSmR3umnI21PSsRfT5vZkKSM\nP8pod//EzLoDz5jZh7TgzyDuSZW7N1hR3cwmAo9Hzns9Mki8m7tvTVQcZjYM6A+8Y2ZGuJv4TTMb\n5e6b4x1HY7FExXQZ4a7TL7fG/RuxHjg66nVfAly/MfJY6VHgfnefF1AYo4HzzeyrQHugk5nNcffv\nBBDLOsI9qm9EXj8KBDX4eQTwkrtvAzCzxwkvS6Wk6lNJ9e8pRpvMrKe7bzKzI4FWaRvjpYG2I6W+\nA4C77zKzBYT/vaVK/PW1mfcDG1MkfgDc/ZPIzy1m9gThx/nN/jNI9OO/J4gkDmZ2HJDbGglVY9z9\nfXc/0t2PcfcBhP/HdUprJVSHY58WGzy/TrHBRHgdGGhm/SIzuS4mXOQwKPcAS939tqACcPefu/vR\n7n4M4f8ezweUUBHpdl4b+bcCMJbgBs9/CJxmZu0iv4yMJaBB80ks2f49NcdniqASjvuyyP5/AUH9\nktNU9bUdKfEdzKygdlZZ5PH+mcBbpEj8DbSZ3wb+QQrED2BmeZGeTsysA3AW8B4t+DOIe0/VYdwL\n3GNm7wEVQCD/s6rDCfYRzx3UU2wwETd292oz+wnhGYhZwGx3D2p22WjgUuC9yNgCB37u7k8FEU8S\nuZpwJe9c4GPgu0EE4e7vmNkc4E2gmnCj/8cgYklWyfTvqTms/iKoNwGPmNnlhGdIfyO4CBvXUNsB\nzAAeToHv0Av4S+SXlSzCvW3PRb5LKsTfkJtInfh7An+PPDLOAR5w92fM7A2a+R1U/FNEREQkDhJe\n/FNEREQkHSmpEhEREYkDJVUiIiIicaCkSkRERCQOlFSJiIiIxIGSKhEREZE4UFIlIiJJz8y6mtlb\nkXUwPzGzdVGvm1Rz0cxmm9mgw5xzpZl9Mz5R13v9/4gq6CtpRnWqREQkpZjZ/wfscfff1vOeeRL/\nj/Hvu0AAAAM/SURBVC2yhMujAS7FJa1IPVUiIpJqDq6CYWbHmlnIzP5qZu8DR5rZ3Wa2xMzeM7Mb\nos79PzMbbmbZZrbdzG40s7fN7CUzK4ic8yszuzrq/BvN7DUzW2Zmp0WO55nZo2b2vpk9Ymavm9nw\nQ4I0uyUS29uR65xBeJ3X30Z62I42s4Fm9lTkGiVmNjDy2fvN7C4ze8PMPogsaYaZDYt8t9LIdfu3\n2n9labZEL1MjIiISb4OBb7n7WwBmVuzuO8wsG1hkZo+6+wd1PpMPLHL3KWb2G+By4Ob6Lu7up5rZ\neYSX8DkHuAr4xN2/Hkmm3qz7GTPrAZzj7kMjrztHLZj8iLs/GTn+PHCFu680s88DvwfGRS7T191H\nRB4XPmtmxwJXAre4+yOR5auCXGZN6lBSJSIiqe7ftQlVxKWR9dpyCK+tNwSom1SVu/szkf03gTMa\nuPbjUef0i+yfQXhtO9z9XTML1fO5bUC1mf0RWADMr3tCZCHl04DHImv/wWefID0cucfyyLqMg4CX\ngf8X6aF63N3/3UDcEgA9/hMRkVS3t3Yn8vjsaqDI3U8Cngba1fOZA1H71TTcyVDRhHMO6S1y9yr+\n/3bun7WKIArD+HNAsUgwtWCTIgEL8Q9+kDSSYJ/0ySfIZ0gaLcWAndgoWthpFywsTCEIFiFFGjU2\nQfG1uINeNtFwYcBceH6wsOzOzG75cuYwcAd4CiwBz/4y7zDJ7SS32nVjfJnB2CTZaesdAy/alqLO\nCUOVJGnajYeay8BX4FtVXeHPVtq/5kzqDbAMUFXXgWsnFq+aBeaSPAc2gJvt1VH7R5J8Bg6qaqnN\nqUFv1t32fBG4CnyoqvkkH5NsMap+nejl0v/j9p8kadr9rugkeVtVe8Ae8Al4fdq4wf2Z6w5sAw9b\nY/z7dn0ZjJkDnlTVJUYBbr09fww8qKoNRhWnFeB+VW0CF4Ed4F0bu19Vu8AMsJrkR1Xda0c+fAf2\nGfV56ZzwSAVJkibQGuAvJDlu240vgYUkPzt+4xFjDe2aDlaqJEmazCzwauzQ0bWegaqx4jGFrFRJ\nkiR1YKO6JElSB4YqSZKkDgxVkiRJHRiqJEmSOjBUSZIkdWCokiRJ6uAX5JpS92QHQFAAAAAASUVO\nRK5CYII=\n", + "text/plain": [ + "<matplotlib.figure.Figure at 0x7f8040158550>" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "#@test {\"output\": \"ignore\"}\n", + "\n", + "# Use the same input data and parameters as the examples above.\n", + "# We're going to build up a list of the errors over time as we train to display later.\n", + "losses = []\n", + "\n", + "with tf.Session() as sess:\n", + " # Set up all the tensors.\n", + " # The input is the x values with the bias appended on to each x.\n", + " input = tf.constant(x_with_bias)\n", + " # We're trying to find the best fit for the target y values.\n", + " target = tf.constant(np.transpose([y]).astype(np.float32))\n", + " # Let's set up the weights randomly\n", + " weights = tf.Variable(tf.random_normal([2, 1], 0, 0.1))\n", + "\n", + " tf.initialize_all_variables().run()\n", + " \n", + " # mu is the learning rate (step size), so how much we jump from the current spot\n", + " mu = 0.002\n", + " \n", + " # The operations in the operation graph.\n", + " # Compute the predicted y values given our current weights\n", + " yhat = tf.matmul(input, weights)\n", + " # How much does this differ from the actual y?\n", + " yerror = tf.sub(yhat, target)\n", + " # Change the weights by subtracting derivative with respect to that weight\n", + " loss = 0.5 * tf.reduce_sum(tf.mul(yerror, yerror))\n", + " gradient = tf.reduce_sum(tf.transpose(tf.mul(input, yerror)), 1, keep_dims=True)\n", + " update_weights = tf.assign_sub(weights, mu * gradient)\n", + " \n", + " # Repeatedly run the operation graph over the training data and weights.\n", + " for _ in range(training_steps):\n", + " sess.run(update_weights)\n", + "\n", + " # Here, we're keeping a history of the losses to plot later\n", + " # so we can see the change in loss as training progresses.\n", + " losses.append(loss.eval())\n", + "\n", + " # Training is done, compute final values for the graph.\n", + " betas = weights.eval()\n", + " yhat = yhat.eval()\n", + "\n", + "# Show the results.\n", + "fig, (ax1, ax2) = plt.subplots(1, 2)\n", + "plt.subplots_adjust(wspace=.3)\n", + "fig.set_size_inches(10, 4)\n", + "ax1.scatter(x, y, alpha=.7)\n", + "ax1.scatter(x, np.transpose(yhat)[0], c=\"g\", alpha=.6)\n", + "line_x_range = (-4, 6)\n", + "ax1.plot(line_x_range, [betas[0] + a * betas[1] for a in line_x_range], \"g\", alpha=0.6)\n", + "ax2.plot(range(0, training_steps), losses)\n", + "ax2.set_ylabel(\"Loss\")\n", + "ax2.set_xlabel(\"Training steps\")\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "TzIETgHwTexL" + }, + "source": [ + "This code looks very similar to the code above, but without using `l2_loss` or `GradientDescentOptimizer`. Let's look at exactly what it is doing instead.\n", + "\n", + "This code is the key difference:\n", + "\n", + ">`loss = 0.5 * tf.reduce_sum(tf.mul(yerror, yerror))`\n", + "\n", + ">`gradient = tf.reduce_sum(tf.transpose(tf.mul(input, yerror)), 1, keep_dims=True)`\n", + "\n", + ">`update_weights = tf.assign_sub(weights, mu * gradient)`\n", + "\n", + "The first line calculates the L2 loss manually. It's the same as `l2_loss(yerror)`, which is half of the sum of the squared error, so $\\frac{1}{2} \\sum (\\hat{y} - y)^2$. With this code, you can see exactly what the `l2_loss` operation does. It's the total of all the squared differences between the target and our estimates. And minimizing the L2 loss will minimize how much our estimates of $y$ differ from the true values of $y$.\n", + "\n", + "The second line calculates $\\sum{x_i (\\hat{y} - y)}$. What is that? It's the partial derivative of the L2 loss, the same thing as what `gradients(loss, weights)` does in the earlier code. Not sure about that? Let's look at it in more detail. The gradient calculation is going to get the partial derivatives of loss with respect to each of the weights so we can change those weights in the direction that will reduce the loss. L2 loss is $\\frac{1}{2} \\sum (\\hat{y} - y)^2$, where $\\hat{y} = w_2 x + w_1$. So, using the chain rule and substituting in for $\\hat{y}$ in the derivative, $\\frac{\\partial}{\\partial w_i} = \\sum{(\\hat{y} - y)\\, x_i}$. `GradientDescentOptimizer` does these calculations automatically for you based on the graph structure.\n", + "\n", + "The third line is equivalent to `weights -= mu * gradient`, so it subtracts a constant the gradient after scaling by the learning rate (to avoid jumping too far each time, which risks moving in the wrong direction). It's also the same thing that `GradientDescentOptimizer(learning_rate).minimize(loss)` does in the earlier code. Gradient descient updates its first parameter based on the values in the second after scaling by the third, so it's equivalent to the `assign_sub(weights, mu * gradient)`.\n", + "\n", + "Hopefully, this other code gives you a better understanding of what the operations we used previously are actually doing. In practice, you'll want to use those high level operators most of the time rather than calculating things yourself. For this toy example and simple network, it's not too bad to compute and apply the gradients yourself from scratch, but things get more complicated with larger networks." + ] + } + ], + "metadata": { + "colabVersion": "0.3.2", + "colab_default_view": {}, + "colab_views": {}, + "kernelspec": { + "display_name": "Python 2", + "language": "python", + "name": "python2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 2 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython2", + "version": "2.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} |