experiments/STT-tensorflow
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Make kGradientAccumulatorInitialValue a function.

Browse Source 
The presence of absl::bit_cast in the static initializer caused an
initialization-order-fiasco issue when I was working on an unrelated
change.

PiperOrigin-RevId: 252921770
This commit is contained in:
James Ring 2019-06-12 16:15:48 -07:00 committed by TensorFlower Gardener
parent 9cf397a716
commit 41ab825f80
2 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
tensorflow/core/tpu/tpu_embedding_optimization_parameters_utils.cc
View File

@ -252,7 +252,7 @@ Status GetOptimizationAlgorithmStateVariables(
StateVariableSpecification gradient_acc;
gradient_acc.set_name("gradient_accumulators");
gradient_acc.mutable_fill_with_constant()->set_initial_value(
kGradientAccumulatorInitialValue);
GradientAccumulatorInitialValue());
state_variables->push_back(std::move(gradient_acc));
}
if (state_variables->size() > kMaxAuxiliaryParameterCount + 1) {

4
tensorflow/core/tpu/tpu_embedding_optimization_parameters_utils.h
View File

@ -84,7 +84,9 @@ static constexpr int kMaxAuxiliaryParameterCount = 3;
// not no-ops on zero gradients, so we need to distinguish an accumulated
// gradient of zero from one that has been cleared after its gradients have
// already been applied to the parameters and accumulators.
const float kGradientAccumulatorInitialValue = absl::bit_cast<float, uint32>(1);
inline float GradientAccumulatorInitialValue() {
return absl::bit_cast<float, uint32>(1);
}
// Computes registration data for per table load Op. Each load Op transfers
// the embedding parameters from the host memory to the TPU memory.