experiments/STT-tensorflow
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Add tests to verify 0.0 is quantized correctly

Browse Source 
We should consider both signed and narrow_range cases.

PiperOrigin-RevId: 266167366
This commit is contained in:
Feng Liu 2019-08-29 10:08:46 -07:00 committed by TensorFlower Gardener
parent 4aa7af5ed4
commit 8a94f84e7e
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
third_party/mlir/lib/Dialect/QuantOps/Utils/FakeQuantSupport.cpp vendored
View File

@ -76,7 +76,7 @@ mlir::quant::fakeQuantAttrsToType(Location loc, unsigned numBits, double rmin,
// points and dequantized to 0.0. // points and dequantized to 0.0.
if (std::fabs(rmax - rmin) < std::numeric_limits<double>::epsilon()) { if (std::fabs(rmax - rmin) < std::numeric_limits<double>::epsilon()) {
return UniformQuantizedType::getChecked(flags, storageType, expressedType, return UniformQuantizedType::getChecked(flags, storageType, expressedType,
1.0, 0, qmin, qmax, loc); 1.0, qmin, qmin, qmax, loc);
} }
// Determine the scale. // Determine the scale.