From ab48dbd4ac2095548a5bc8505e08e751d409727f Mon Sep 17 00:00:00 2001 From: "A. Unique TensorFlower" Date: Fri, 26 Feb 2016 16:23:49 -0800 Subject: Fixing operator order in LRN docs to match code. The implementation adds the bias to a temporary, that temporary is what is then the base with exponent beta. The implementation also agrees with the equation in Section 3.3 of the referenced Krizhevsky et. al. paper. Change: 115721267 --- tensorflow/core/ops/nn_ops.cc | 2 +- tensorflow/core/ops/ops.pbtxt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tensorflow/core/ops/nn_ops.cc b/tensorflow/core/ops/nn_ops.cc index 1d1afe61d3..2a2edfebd3 100644 --- a/tensorflow/core/ops/nn_ops.cc +++ b/tensorflow/core/ops/nn_ops.cc @@ -349,7 +349,7 @@ each component is divided by the weighted, squared sum of inputs within sqr_sum[a, b, c, d] = sum(input[a, b, c, d - depth_radius : d + depth_radius + 1] ** 2) - output = input / (bias + alpha * sqr_sum ** beta) + output = input / (bias + alpha * sqr_sum) ** beta For details, see [Krizhevsky et al., ImageNet classification with deep convolutional neural networks (NIPS 2012)] diff --git a/tensorflow/core/ops/ops.pbtxt b/tensorflow/core/ops/ops.pbtxt index 585dd09851..d6f4a2f780 100644 --- a/tensorflow/core/ops/ops.pbtxt +++ b/tensorflow/core/ops/ops.pbtxt @@ -4137,7 +4137,7 @@ op { description: "An exponent." } summary: "Local Response Normalization." - description: "The 4-D `input` tensor is treated as a 3-D array of 1-D vectors (along the last\ndimension), and each vector is normalized independently. Within a given vector,\neach component is divided by the weighted, squared sum of inputs within\n`depth_radius`. In detail,\n\n sqr_sum[a, b, c, d] =\n sum(input[a, b, c, d - depth_radius : d + depth_radius + 1] ** 2)\n output = input / (bias + alpha * sqr_sum ** beta)\n\nFor details, see [Krizhevsky et al., ImageNet classification with deep\nconvolutional neural networks (NIPS 2012)]\n(http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks)." + description: "The 4-D `input` tensor is treated as a 3-D array of 1-D vectors (along the last\ndimension), and each vector is normalized independently. Within a given vector,\neach component is divided by the weighted, squared sum of inputs within\n`depth_radius`. In detail,\n\n sqr_sum[a, b, c, d] =\n sum(input[a, b, c, d - depth_radius : d + depth_radius + 1] ** 2)\n output = input / (bias + alpha * sqr_sum) ** beta\n\nFor details, see [Krizhevsky et al., ImageNet classification with deep\nconvolutional neural networks (NIPS 2012)]\n(http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks)." } op { name: "LRNGrad" -- cgit v1.2.3