experiments/STT-tensorflow
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Make kGradientAccumulatorInitialValue a function.

Browse Source 
The presence of absl::bit_cast in the static initializer caused an
initialization-order-fiasco issue when I was working on an unrelated
change.

PiperOrigin-RevId: 252921770
This commit is contained in:
James Ring 2019-06-12 16:15:48 -07:00 committed by TensorFlower Gardener
parent 9cf397a716
commit 41ab825f80
2 changed files with 4 additions and 2 deletions

2
tensorflow/core/tpu/tpu_embedding_optimization_parameters_utils.cc
View File

@ -252,7 +252,7 @@ Status GetOptimizationAlgorithmStateVariables(
StateVariableSpecification gradient_acc; StateVariableSpecification gradient_acc;
gradient_acc.set_name("gradient_accumulators"); gradient_acc.set_name("gradient_accumulators");
gradient_acc.mutable_fill_with_constant()->set_initial_value( gradient_acc.mutable_fill_with_constant()->set_initial_value(
kGradientAccumulatorInitialValue); GradientAccumulatorInitialValue());
state_variables->push_back(std::move(gradient_acc)); state_variables->push_back(std::move(gradient_acc));
} }
if (state_variables->size() > kMaxAuxiliaryParameterCount + 1) { if (state_variables->size() > kMaxAuxiliaryParameterCount + 1) {

4
tensorflow/core/tpu/tpu_embedding_optimization_parameters_utils.h
View File

@ -84,7 +84,9 @@ static constexpr int kMaxAuxiliaryParameterCount = 3;
// not no-ops on zero gradients, so we need to distinguish an accumulated // not no-ops on zero gradients, so we need to distinguish an accumulated
// gradient of zero from one that has been cleared after its gradients have // gradient of zero from one that has been cleared after its gradients have
// already been applied to the parameters and accumulators. // already been applied to the parameters and accumulators.
const float kGradientAccumulatorInitialValue = absl::bit_cast<float, uint32>(1); inline float GradientAccumulatorInitialValue() {
return absl::bit_cast<float, uint32>(1);
}
// Computes registration data for per table load Op. Each load Op transfers // Computes registration data for per table load Op. Each load Op transfers
// the embedding parameters from the host memory to the TPU memory. // the embedding parameters from the host memory to the TPU memory.