Updating tutorial to use sparse_softmax_cross_entropy()

As per request from @MarkDaoust, I have updated the tutorial section of the notebook to reflect the usage of tf.losses.sparse_softmax_cross_entropy() rather than tf.losses.softmax_cross_entropy() using one-hot vectors.
This commit is contained in:
Dan Douthit 2018-06-04 16:08:38 -06:00 committed by GitHub
parent 4196890b1e
commit d7c971c156
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -169,8 +169,7 @@ def cnn_model_fn(features, labels, mode):
return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions)
# Calculate Loss (for both TRAIN and EVAL modes)
onehot_labels = tf.one_hot(indices=tf.cast(labels, tf.int32), depth=10)
loss = tf.losses.sparse_softmax_cross_entropy(labels=onehot_labels, logits=logits)
loss = tf.losses.sparse_softmax_cross_entropy(labels=labels, logits=logits)
# Configure the Training Op (for TRAIN mode)
if mode == tf.estimator.ModeKeys.TRAIN:
@ -471,50 +470,24 @@ as the loss metric. The following code calculates cross entropy when the model
runs in either `TRAIN` or `EVAL` mode:
```python
onehot_labels = tf.one_hot(indices=tf.cast(labels, tf.int32), depth=10)
loss = tf.losses.softmax_cross_entropy(
onehot_labels=onehot_labels, logits=logits)
loss = tf.losses.sparse_softmax_cross_entropy(labels=labels, logits=logits)
```
Let's take a closer look at what's happening above.
Our `labels` tensor contains a list of predictions for our examples, e.g. `[1,
9, ...]`. In order to calculate cross-entropy, first we need to convert `labels`
9, ...]`. By using `tf.losses.sparse_softmax_cross_entropy()` we do not need to convert `labels`
to the corresponding
[one-hot encoding](https://www.quora.com/What-is-one-hot-encoding-and-when-is-it-used-in-data-science):
[one-hot encoding](https://www.quora.com/What-is-one-hot-encoding-and-when-is-it-used-in-data-science)
that is commonly used in machine learning applications.
```none
[[0, 1, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 1],
...]
```
We use the @{tf.one_hot} function
to perform this conversion. `tf.one_hot()` has two required arguments:
* `indices`. The locations in the one-hot tensor that will have "on
values"—i.e., the locations of `1` values in the tensor shown above.
* `depth`. The depth of the one-hot tensor—i.e., the number of target classes.
Here, the depth is `10`.
The following code creates the one-hot tensor for our labels, `onehot_labels`:
```python
onehot_labels = tf.one_hot(indices=tf.cast(labels, tf.int32), depth=10)
```
Because `labels` contains a series of values from 09, `indices` is just our
`labels` tensor, with values cast to integers. The `depth` is `10` because we
have 10 possible target classes, one for each digit.
Next, we compute cross-entropy of `onehot_labels` and the softmax of the
predictions from our logits layer. `tf.losses.softmax_cross_entropy()` takes
`onehot_labels` and `logits` as arguments, performs softmax activation on
Next, we compute cross-entropy of `labels` and the softmax of the
predictions from our logits layer. `tf.losses.sparse_softmax_cross_entropy()` takes
`labels` and `logits` as arguments, performs softmax activation on
`logits`, calculates cross-entropy, and returns our `loss` as a scalar `Tensor`:
```python
loss = tf.losses.softmax_cross_entropy(
onehot_labels=onehot_labels, logits=logits)
loss = tf.losses.sparse_softmax_cross_entropy(labels=onehot_labels, logits=logits)
```
### Configure the Training Op