experiments/STT-tensorflow
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Fixing operator order in LRN docs to match code.

Browse Source 
The implementation adds the bias to a temporary, that temporary is what is then the base with exponent beta. The implementation also agrees with the equation in Section 3.3 of the referenced Krizhevsky et. al. paper.
Change: 115721267
This commit is contained in:
A. Unique TensorFlower 2016-02-26 16:23:49 -08:00 committed by TensorFlower Gardener
parent 7552e9e561
commit ab48dbd4ac
2 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
tensorflow/core/ops/nn_ops.cc
View File

@ -349,7 +349,7 @@ each component is divided by the weighted, squared sum of inputs within
sqr_sum[a, b, c, d] =
sum(input[a, b, c, d - depth_radius : d + depth_radius + 1] ** 2)
output = input / (bias + alpha * sqr_sum ** beta)
output = input / (bias + alpha * sqr_sum) ** beta
For details, see [Krizhevsky et al., ImageNet classification with deep
convolutional neural networks (NIPS 2012)]

2
tensorflow/core/ops/ops.pbtxt
View File

@ -4137,7 +4137,7 @@ op {
description: "An exponent."
}
summary: "Local Response Normalization."
description: "The 4-D `input` tensor is treated as a 3-D array of 1-D vectors (along the last\ndimension), and each vector is normalized independently. Within a given vector,\neach component is divided by the weighted, squared sum of inputs within\n`depth_radius`. In detail,\n\n sqr_sum[a, b, c, d] =\n sum(input[a, b, c, d - depth_radius : d + depth_radius + 1] ** 2)\n output = input / (bias + alpha * sqr_sum ** beta)\n\nFor details, see [Krizhevsky et al., ImageNet classification with deep\nconvolutional neural networks (NIPS 2012)]\n(http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks)."
description: "The 4-D `input` tensor is treated as a 3-D array of 1-D vectors (along the last\ndimension), and each vector is normalized independently. Within a given vector,\neach component is divided by the weighted, squared sum of inputs within\n`depth_radius`. In detail,\n\n sqr_sum[a, b, c, d] =\n sum(input[a, b, c, d - depth_radius : d + depth_radius + 1] ** 2)\n output = input / (bias + alpha * sqr_sum) ** beta\n\nFor details, see [Krizhevsky et al., ImageNet classification with deep\nconvolutional neural networks (NIPS 2012)]\n(http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks)."
}
op {
name: "LRNGrad"