Imported from GitHub PR https://github.com/tensorflow/tensorflow/pull/37679
This PR enables CUDNN CTC Loss to support both deterministic and non-deterministic algos.
If determinism is required, we will stick to deterministic algo.
Otherwise, the faster non-deterministic algo will be tried in cudnnGetCtcLossWorkspace(). If it fails, we fall-back to the deterministic algo.
fyi @nluehr @sanjoy
Copybara import of the project:
--
e338f0bad8a10c6b0e6284e609dfe2b6ebeab9f0 by Kaixi Hou <kaixih@nvidia.com>:
CTC loss support two algos
--
c90ebbd533202c791e342b5716c3410c4557ea1a by Kaixi Hou <kaixih@nvidia.com>:
Reuse macros of cudnn launch
--
c9fd60fa4abc37674923acb802e46a7ac8055796 by Kaixi Hou <kaixih@nvidia.com>:
Add unittest for ctc_loss fallback algo
--
e29100d0202e2a519845b818d6f9603505d2c83e by Kaixi Hou <kaixih@nvidia.com>:
Shorten the comment sentence
COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/tensorflow/pull/37679 from houtoms:ctc_loss_workspace_check e29100d0202e2a519845b818d6f9603505d2c83e
PiperOrigin-RevId: 305942210
Change-Id: I57062bcb5f04097a3280eb6eeb5de51bae6ef3ca
Imported from GitHub PR https://github.com/tensorflow/tensorflow/pull/36267
This PR reverts ROCm to use MIOpen Find Mode APIs (be default) for convolution. The use of MIOpen Immediate Mode API (instead of the Find Mode APIs) can be specified by the setting the env var `TF_ROCM_USE_IMMEDIATE_MODE=1`.
Almost all of the changes in this PR are within code that is specific to the ROCm platform, so this PR should not have any impact on non ROCm builds.
----------------
/cc @chsigg @whchung
Copybara import of the project:
--
5675e37e5f9b595dab45f44239cbfab222e9dcc2 by Deven Desai <deven.desai.amd@gmail.com>:
Renaming MIMIC_FIND_MODE to RETURN_BEST_ALGO_ONLY. This is being done as preparation for the implementation to re-insert calls to Find Mode API. MIMIC_FIND_MODE was a poor name for what it was doing, and would have resulted in confusion once Find Mode APIs are re-inserted. This commit also simplifies the implementation associated with RETURN_BEST_ALGO_ONLY
--
5fe0ad377dc7e333acf8aac91e3333781242fe5c by Deven Desai <deven.desai.amd@gmail.com>:
changes to fix compile time warnings in rocm_dnn.cc
--
e3dcc169353646c4b5e684b7398cf1db743079cb by Deven Desai <deven.desai.amd@gmail.com>:
Making the implementation of the Conv3D Gradient kernels consistent with the implementations of all the other Conv2D/3D kernels
--
4d4a5cede3b6e959fcc06fdb6211e4c9ef5343f5 by Deven Desai <deven.desai.amd@gmail.com>:
Updating the convolution kernel implementation(s) to ensure that the AlgorithmConfig::scratch_size field is always populated correctly before it is passed as an argument to the ThenConvolve* routine(s)
--
64ffda476af322ad804d1f5b8d7a05719e2f183c by Deven Desai <deven.desai.amd@gmail.com>:
Using the workspace memory size from the AlgorithmConfig argument, instead of calling an MIOpen API to determine it (during the call the DoPrepareForConvolution)
--
d42a76e177a26124e966c83cbbb809dbdbdcabbe by Deven Desai <deven.desai.amd@gmail.com>:
Updating the ROCm XLA Convolution Algorithm Picker implementation, to use the scratch_size that was returned in the prior call to GetMIOpenAlgorithms. Note that the code to save the scratch_size information in the new custom-call instruction (once the best conv algorithm has been determined) already exists, this commit does not change that part at all. This commit modifies how the scratch_size is determined for RunGpuConv calls that happen during the call to determine the best algorithm for a given convolution
--
416aeccbfc430c71b27cbe04a57dcd1577b34fae by Deven Desai <deven.desai.amd@gmail.com>:
Changes for the TF Convolution Kernel implementation and the Stream Executor DNN layer/api to accomodate support for Find Mode. Putting in empty placeholders in places where the Find Mode implementation will live
--
253664ce7ee59bb2ffbc2b4b3fe94963e54837c1 by Deven Desai <deven.desai.amd@gmail.com>:
Re-inserting the Find Mode Implementation. It is still disabled by default
--
30debc7b11afdbc1651c860b65cdd2fba1b9ba50 by Deven Desai <deven.desai.amd@gmail.com>:
Switching the default to Find Mode
--
b0b670e6ee2eaa6823618d4aa8858846a4cbbd89 by Deven Desai <deven.desai.amd@gmail.com>:
Disabling a subtest that fails because of bug in MIOpen Find Mode. MLOpen Issue #2379 has been filed to trach the bug.
COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/tensorflow/pull/36267 from ROCmSoftwarePlatform:google_upstream_rocm_miopen_find_mode b0b670e6ee2eaa6823618d4aa8858846a4cbbd89
PiperOrigin-RevId: 305424670
Change-Id: Ibd02cd2c43f88e619bd77e996614ded0d96d42d5
The static constexpr datamember declarations in the various ToDataType
specialiations (in stream_executor/dnn.h) do not have a corresponding
definition outside of the class.
This results in compilation failures in debug mode
PrepareForConvolution:
* Turn the public interface into template, non-virtual public function, which calls into
* public, type-erased, virtual function.
Number of overloads is shrinked by the number of primitive types.
PiperOrigin-RevId: 230615784
This is the first of a series patches that log StreamExecutor
convolution calls. This patch introduced structured (proto) logging,
suitable for serialize and potentially deserialize.
PiperOrigin-RevId: 221667516
The DNN library communicates to the caller with algorithm information in
3 ways:
(a) The caller asks DNN, for a list of available algorithms.
(b) When calling a convolution/RNN, the caller asks the DNN library for a suggestion
of an algorithm. This also includes the case that the caller just wants one
specific algorithm.
(c) The response of (b), sent from the DNN library back to the caller,
including which algorithm is picked, and how much scratch space is used.
For (a), the data is (algo_id, tensor_ops_enabled).
For (b), the data is (optional (algo_id, tensor_ops_enabled), optional (algo_id, tensor_ops_enable) with no scratch).
For (c), the data is (selected (algo_id, tensor_ops_enabled), the actual scratch size).
Ideally we can have separate data structures for (a), (b) and (c), e.g.
AlgorithmDesc for (a), AlgorithmConfig for (b), and ProfileResult for
(c).
Currently, the scratch size is in AlgorithmDesc, and the optionality is
in AlgorithmDesc as well.
This patch moves scratch size to ProfileResult, and optionality to
AlgorithmConfig.
Historically, PR20708 first tried to add the scratch size to AlgorithmConfig,
but that's not the right place (as explained above). The scratch size ended up
in AlgorithmDesc, because I was concerned that changing ProfileResult was too much
work (I asked for a TODO for it, but I didn't find it in the current code).
Now hopefully all bits are moved to right places.
PiperOrigin-RevId: 219359118
CuDNN supports both convolution (CUDNN_CONVOLUTION) and cross correlation
(CUDNN_CROSS_CORRELATION). However, only the latter was hooked up, causing
Tensorflow error: Status: Hit a case for convolution that is not implemented on GPU.
for convolutions of the first kind (corresponding to convolutions with both
window dimensions reversed at the HLO level). Reversing the dimensions (i.e.
doing convolutions rather than cross correlations) is the default behavior
for the Flux.jl ML framework, so it's easy to hit this error trying to run
pre-existing Flux models through the Julia:XLA->XLA:GPU compilation path.
Plumb through the reversal option to CuDNN to make this pattern work. The
same HLO already works fine against the CPU and TPU backends.
CudnnRNN V2 kernels run all applicable cudnn rnn algorithms and pick the best one for following runs.
* To enable autotune, TF_CUDNN_RNN_USE_AUTOTUNE and TF_CUDNN_RNN_USE_V2 need to be set to {"1" or unset}.
* TF_CUDNN_RNN_USE_AUTOTUNE does not work with existing CudnnRNN kernels.
* V2 kernels work with existing cudnn checkpoints, since it doesn't change persistence format.
This change
* Introduces v2 kernels as templates inheriting the v1 kernels.
* Profiles fwd and bak runs in v2 kernel (forward pass)
* Exposes the chosen algorithm as fwd op output and bak op input.
* Changes rnn descriptor cache key to include AlgorithmDesc (since cudnn rnn descriptor can't be reused across different algorithms)
* Updates unittests s.t. it tests both v1 and v2 kernels. When testing v2 kernels, autotune is turned on.
PiperOrigin-RevId: 194333948
Step 1 of re-namespace'ing StreamExecutor into ::stream_executor.
This moves everything inside of stream_executor/..., and leaves a
namespace alias into ::perftools::gputools. The next steps will clean
up users to use the new namespace.
This is mostly a mechanical change, but it also includes a bunch of
non-mechanical changes that ideally would be split out into separate
patches. Unfortunately they all sort of need to be shoved in here for
various reasons:
- forward declarations need to be in the same namespace as the actual
types, so we need to change all forward declarations of
StreamExecutor types in this one patch.
- Uses of these forward declarations need to be changed to the new
namespace (or otherwise we need to add a namespace alias to the
relevant header, but this is pretty ugly).
- Various initialization code needs to live in StreamExecutor's "real"
namespace, so all this needs to be changed.
PiperOrigin-RevId: 193256128
END_PUBLIC
---
Commit ee0fdc296 authored by Gunhan Gulsoy<gunan@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Add noasan tag to estimator_test
PiperOrigin-RevId: 171075499
---
Commit a02116882 authored by Justin Lebar<jlebar@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
[XLA:CPU] Put the HLO name in IR values that hold the HLO's value.
PiperOrigin-RevId: 171075449
---
Commit 89aaac4bc authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Allow Layer.add_update() in Eager mode.
PiperOrigin-RevId: 171070861
---
Commit 840dcae57 authored by Amit Patankar<amitpatankar@google.com>
Committed by gunan<gunan@google.com>:
Updating the install sources file with a supported configs table (#13450)
* Updating the install sources file with a supported configs page.
* Implementing Gunan's suggestions.
* Adding GCC string to Linux compiler.
* Updating the bazel/cmake column.
---
Commit 89df2e336 authored by Igor Saprykin<isaprykin@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Add the 'is_the_final_export' signal to Exporters. Use them in training.
When the training ends, the final export is performed via `Exporter.export()` call. That final export is going to have is_the_final_export parameter being set to true.
If `TrainSpec.max_steps` is `None`, then "when training ends" is undefined. We are going to train forever. In that case, `is_the_final_export` is going to be always False. I added a note about it.
PiperOrigin-RevId: 171070760
---
Commit 4486b4f69 authored by Akshay Agrawal<akshayka@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Make graph_callable compatible with functions that do not return anything
PiperOrigin-RevId: 171067061
---
Commit 39565c0cb authored by Martin Wicke<wicke@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Publish train_and_evaluate and associated classes.
PiperOrigin-RevId: 171066379
---
Commit 3b4477000 authored by Saurabh Saxena<srbs@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Make VariantTensorData::tensors_size() const.
PiperOrigin-RevId: 171063397
---
Commit 53cc63a2d authored by Dhananjay Nakrani<dhananjayn@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
[part 1] Add support for int32 & int64 in RandomPoissonOp.
This computes int32/int64-precision poisson samples with double precision intermediate calculations (same as it's done for `half`) respectively.
part 2 will switch over python calls to new op once forward compatibility period has passed.
PiperOrigin-RevId: 171058336
---
Commit 70fc9bf9b authored by Asim Shankar<ashankar@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Java: Add support for loading op libraries dynamically.
This change adds the equivalent of tf.load_op_library in Python to Java.
(5c7f9e316d
was required to make this possible)
Though, TensorFlow.loadLibrary() is likely to fail on Windows as symbols
required by custom op libraries (those exported by the tensorflow_framework library)
are not exported by the monolithic JNI library yet.
This should help with #10454 and #13476
PiperOrigin-RevId: 171054707
---
Commit e7c53698e authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Internal cleanup
PiperOrigin-RevId: 171053770
---
Commit cc8ee6c0f authored by Alexandre Passos<apassos@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Fast path for tf.conj when it should be pass-through.
PiperOrigin-RevId: 171053662
---
Commit c41dbc3c1 authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Adding TF Boosted trees regression example on boston dataset, minor fix for mnist example.
PiperOrigin-RevId: 171052367
---
Commit d66e77f7c authored by Mustafa Ispir<ispir@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Added get variable utils to tf.estimator.Estimator.
PiperOrigin-RevId: 171052121
---
Commit 083bd5dde authored by Asim Shankar<ashankar@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Java: Add support for loading op libraries dynamically.
This change adds the equivalent of tf.load_op_library in Python to Java.
(5c7f9e316d
was required to make this possible)
Though, TensorFlow.loadLibrary() is likely to fail on Windows as symbols
required by custom op libraries (those exported by the tensorflow_framework library)
are not exported by the monolithic JNI library yet.
This should help with #10454 and #13476
PiperOrigin-RevId: 171054707
---
Commit 2fe6cf285 authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Internal cleanup
PiperOrigin-RevId: 171053770
---
Commit 15155493b authored by Alexandre Passos<apassos@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Fast path for tf.conj when it should be pass-through.
PiperOrigin-RevId: 171053662
---
Commit 6c954d0b3 authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Adding TF Boosted trees regression example on boston dataset, minor fix for mnist example.
PiperOrigin-RevId: 171052367
---
Commit ad69076eb authored by Mustafa Ispir<ispir@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Added get variable utils to tf.estimator.Estimator.
PiperOrigin-RevId: 171052121
---
Commit 3cf41b2ed authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Test save/restore variable from graph_callable.
PiperOrigin-RevId: 171051237
---
Commit cf17ec96e authored by Yangzihao Wang<yangzihao@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Add V2 versions of output window size computation functions for convolution.
These V2 versions take arbitrary dilation rates.
In preparation for the support of native cudnn dilated convolution.
PiperOrigin-RevId: 171048878
---
Commit 491584ff4 authored by Asim Shankar<ashankar@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
eager: Always run dataset iterator operations on CPU.
It has no kernels for other devices.
With an explicit "tf.device()" before invoking the kernel we ensure
that Iterator.next() functions even when placed inside a:
with tf.device("/device:GPU:0")
PiperOrigin-RevId: 171048558
---
Commit 3b354016e authored by Igor Saprykin<isaprykin@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Rename SavedModelExporter to LatestExporter.
PiperOrigin-RevId: 171048345
---
Commit 943c6d7af authored by Jianwei Xie<xiejw@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
errors out if the evaluator has task id > 0.
PiperOrigin-RevId: 171047652
---
Commit 8c9ef4466 authored by Mark Heffernan<meheff@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Expand set of 64-bit type tests in LocalClientExecuteTest.ShapeBufferToLiteralConversion64bit and factor out into their own test.
PiperOrigin-RevId: 171043047
---
Commit cc521eb06 authored by Benoit Steiner<bsteiner@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Place all the nodes created by the trivial_test_graph_input_yielder
PiperOrigin-RevId: 171045878
---
Commit 9b9301240 authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
[XLA:CPU] Factor out parallel task assignment from cpu parallelization prep (no functional changes).
PiperOrigin-RevId: 171045137
---
Commit 558d878d9 authored by Allen Lavoie<allenl@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
TFTS: Move normalization to the base class, start using it for state space models
Preivously, state space models adjusted their priors based on the data
(e.g. setting initial variances to match sample variance) but did not normalize
the data itself. When the data has a rather extreme scale, this runs into
precision issues. After this CL, state space models will first normalize, then
use adjusted statistics on top of that normalization to estimate initial
observation/transition noise.
Also fixes an issue where start-of-series statistics were incorrect for the first
batch (which only shows up with large input scales).
PiperOrigin-RevId: 171044863
---
Commit 266f77156 authored by Mark Heffernan<meheff@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Expand set of 64-bit type tests in LocalClientExecuteTest.ShapeBufferToLiteralConversion64bit and factor out into their own test.
PiperOrigin-RevId: 171043047
---
Commit c9915d1a2 authored by Shanqing Cai<cais@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
[tf-signal] Fix pip tests by including test_util in signal_py
PiperOrigin-RevId: 171042732
---
Commit f8550f4e9 authored by Mark Heffernan<meheff@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Expand set of 64-bit type tests in LocalClientExecuteTest.ShapeBufferToLiteralConversion64bit and factor out into their own test.
PiperOrigin-RevId: 171043047
---
Commit 87dc532cd authored by Shanqing Cai<cais@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
[tf-signal] Fix pip tests by including test_util in signal_py
PiperOrigin-RevId: 171042732
---
Commit 0578dd65e authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Add more debugging output for XLA send/recv.
PiperOrigin-RevId: 171041978
---
Commit 23992bb09 authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Several minor documentation fixes.
PiperOrigin-RevId: 171038610
---
Commit af14ed3f3 authored by Jianwei Xie<xiejw@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Some docstring twists and argument validations.
PiperOrigin-RevId: 171037949
---
Commit 6b90a65f6 authored by Mark Heffernan<meheff@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Remove "hybrid" HloModuleConfig option. The option was used to generate executables which only generated the array values of tuple-shaped outputs, not the tuple index tables.. With cl/170133015, ShapedBuffers which hold the computation output now have materialized tuples with these index tables so this option is no longer desired or necessary.
No functional change. Just cleanup.
PiperOrigin-RevId: 171035738
---
Commit 41a0264ab authored by Mustafa Ispir<ispir@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Added utilities to make global step reading deterministic. Used them in Estimator.
Enabled/Fixed some tests.
PiperOrigin-RevId: 171035291
---
Commit 9d7843c0a authored by Skye Wanderman-Milne<skyewm@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Add optional unused_input_map_keys output param to ImportGraphDef
This is a more general feature than that in the Python importer, which
raises an exception if the input map contains unused names.
PiperOrigin-RevId: 171029316
---
Commit 4f10a6597 authored by Mark Heffernan<meheff@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Add vlogging of HloModule before and after fusion.
PiperOrigin-RevId: 171029054
---
Commit 9e658545a authored by Reed Wanderman-Milne<reedwm@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Document what dtype tf.image.resize_images returns.
For consistency, tf.image.resize_images now will always return a float32 when method != ResizeMethod.NEAREST_NEIGHBOR. Before, it returned the same dtype as its input if it could be determined statically that the height and width would not be changed.
PiperOrigin-RevId: 171028825
---
Commit 4d70239f0 authored by Jianwei Xie<xiejw@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Replace the contrib FC with core FC in canned Estimator docstring.
PiperOrigin-RevId: 171027602
---
Commit 6a1b867ff authored by Jianwei Xie<xiejw@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Adds the docstring with details for tf.estimator.train_and_evaluate
PiperOrigin-RevId: 171027527
---
Commit 7209c1602 authored by Peter Hawkins<phawkins@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
[TF:XLA] Mark IdentityN as CompilationOnly().
PiperOrigin-RevId: 171025171
---
Commit 8e22eb874 authored by FAIJUL<md.faijul.amin@intel.com>
Committed by Benoit Steiner<benoitsteiner@users.noreply.github.com>:
Eigen BiasAdd and BiasAddGrad Fix for NCHW Format. (#13158)
---
Commit 7db7a890c authored by Jingyue Wu<jingyue@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
[Grappler] Move InferOutputShapes to GraphProperties.
So it can be used by other optimizers. No functional changes.
PiperOrigin-RevId: 171010106
---
Commit 2114fd51e authored by Peter Hawkins<phawkins@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
[TF:XLA] Improve numerical stability of SoftPlus.
PiperOrigin-RevId: 171003559
---
Commit 727d6270f authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Fix race condition in TensorForest tree traversal.
PiperOrigin-RevId: 170990425
---
Commit d016cb020 authored by Suharsh Sivakumar<suharshs@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Fix c++ gradients issue where multiple dependent outputs result in incorrect answer.
The issue is that we incorrectly calculate the pending num_expected_backprops for outputs nodes when one output transitively depends on another. this is because we use output nodes as an indicator of when we need to end our traversal. Instead we should only use output nodes that don't transitively get consumed by other output nodes as end indicators for our traversal. This change implements that fix.
Fixes#13190
PiperOrigin-RevId: 170971937
---
Commit 5405f3bd7 authored by gunan<gunan@google.com>
Committed by Frank Chen<frankchn@gmail.com>:
Fix tf-signal tests on pip packages. (#13483)
---
Commit f9f037c1c authored by Eugene Brevdo<ebrevdo@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Bugfix to LSTMBlockCell and friends: clipping is off by default.
* Rename broken API argu clip_cell boolean to cell_clip value.
* Make default no clipping.
PiperOrigin-RevId: 170960975
---
Commit bfaaefa9e authored by Frank Chen<frankchn@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Update APIs for TPU Cluster Resolver to remove the custom API definition and instead use a standard definition file stored in GCS.
PiperOrigin-RevId: 170960877
---
Commit c31c118a3 authored by Ian Langmore<langmore@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Extend tf.contrib.bijector API to handle some non-injective transforms.
AbsoluteValue Bijector added to contrib/distributions/bijectors/
TransformedDistribution udpated to handle some non-injective transforms.
PiperOrigin-RevId: 170960054
---
Commit 664dd0859 authored by Frank Chen<frankchn@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Disable cluster_function_library_runtime_test on Mac OS as it is currently failing with an Unimplemented error
PiperOrigin-RevId: 170958505
---
Commit 6af7ab97a authored by Mahmoud Abuzaina<mahmoud.abuzaina@intel.com>
Committed by gunan<gunan@google.com>:
MKL-DNN open source integration. (#13135)
* MKL-DNN conv and build integration
* Adding new files that were mistakenly missing from the PR
* Minor change in the pip package build file
* Added missing #include
* Fixed a linking failure when running the bazel test
* Fixing BUILD file format
* Using -fopenmp for building mkl_dnn only when running on linux
* Fixing build rule attribute value
* Removing unnecessary deps from mkl test rule
* Removed deps on mkl-dnn when not building with --config=mkl
---
Commit 93fa1af76 authored by Akshay Agrawal<akshayka@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Make graph_callable, defun tf_decorators
PiperOrigin-RevId: 170948777
---
Commit b39525785 authored by Mustafa Ispir<ispir@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Added comment re:behavior of listener in case of multiple saver hooks.
PiperOrigin-RevId: 170946536
---
Commit de14fcbb6 authored by Igor Saprykin<isaprykin@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Support evaluation in `_TrainingExecutor.run_master()`.
This CL aims to address the following TODO:
# TODO(b/66720832): Once listener API is added into Estimator.train, the
# eval and export process should be wrapped as a listener and passed to
# _start_distributed_training. The expected behavior should be
# 1. The export is invoked after each intermediate evaluation.
# 2. The evaluation and export should be invoked correctly at the end of
# training. This should be fine if the listener works as intended (it will
# send the `after_save` signal for the final ckpt saving).
1. is achieved as follows:
a. saving_evaluators are added to the CheckpointSaverHook's listeners inside the Estimator.
b. MonitoredSession calls after_run() of CheckpointSaverHook, which in turn calls after_save on the listeners.
2. is achieved in a similar way, but when MonitoredSession calls .end() on CheckpointSaverHook.
PiperOrigin-RevId: 170945961
---
Commit d4ea993ca authored by Alexandre Passos<apassos@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Removes unnecessary eager-mode call to convert_to_tensor in record_gradient.
PiperOrigin-RevId: 170944265
---
Commit add6d2d03 authored by RJ Ryan<rjryan@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
[tf-signal] Use tf.spectral.dct in mfccs_from_log_mel_spectrograms instead of a private implementation.
PiperOrigin-RevId: 170943986
---
Commit b959da92f authored by Jiri Simsa<jsimsa@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Fixing CPU implementation of parallel_stack for tensors with non-zero rank.
PiperOrigin-RevId: 170942814
---
Commit 4cf61262a authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Improve TFGAN documentation.
PiperOrigin-RevId: 170940188
---
Commit 0068086b9 authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Introduce `tf.data` namespace.
PiperOrigin-RevId: 170939033
---
Commit 0c8dbc1fd authored by Alexandre Passos<apassos@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
matmul uses shape_tuple internally
PiperOrigin-RevId: 170938790
---
Commit ad37fa81f authored by Igor Saprykin<isaprykin@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Refactor ExportStrategies into Exporters.
This design eliminates some indirection. Instead of combining an `export_fn` with `make_export_strategy` call to arrive at an ExportStrategy that is going to call the supplied `export_fn` inside its `export` call with Exporters one just defines the `export` call in an Exporter.
PiperOrigin-RevId: 170936640
---
Commit b925f8553 authored by Alexandre Passos<apassos@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Fast-path for EagerTensorBase.dtype
PiperOrigin-RevId: 170933005
---
Commit 08e266d9b authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Pass activity_regularizer to __init__ instead of using the (now
deprecated) property setter.
PiperOrigin-RevId: 170932807
---
Commit b002c8b7d authored by Jingyue Wu<jingyue@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
[Grappler] Fold chains of reshapes.
Reshape(Reshape(input, shape1), shape2) is equivalent to Reshape(input,
shape2).
PiperOrigin-RevId: 170932278
---
Commit 075d1d13b authored by horance<horance@aliyun.com>
Committed by Frank Chen<frankchn@gmail.com>:
remove warning for forward decl (#13459)
---
Commit 931609fcf authored by Ryohei Kuroki<ryohei.kuroki@gmail.com>
Committed by Frank Chen<frankchn@gmail.com>:
Remove unnecessary specification for default kernel name (#13465)
---
Commit 94463f521 authored by Akshay Agrawal<akshayka@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Preserve target function signature in custom_gradient decorator
PiperOrigin-RevId: 170931715
---
Commit 681056636 authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Internal change to simplify prediction ops.
- it no longer returns predictions_no_dropout, which is mostly for debugging purpose.
- as a consequence, MultipleAdditiveTrees::Predict() doesn't return prediction_no_dropout, and it accept trees_to_include indexes intead of trees_to_drop indexes.
PiperOrigin-RevId: 170926422
---
Commit d6e963b82 authored by Asim Shankar<ashankar@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
SYCL: Fix build breakage introduced in
f0e8c545e0Fixes#13350
PiperOrigin-RevId: 170923862
---
Commit 5123f2971 authored by A. Unique TensorFlower<gardener@tensorflow.org>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Internal cleanup.
PiperOrigin-RevId: 170922297
---
Commit d0c76cd18 authored by Igor Saprykin<isaprykin@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Handle the absence of a fresh eval checkpoint in `run_local`.
It is ~unexpected condition for an eval checkpoint to not be available after a train call to the estimator. There is a corner case when it is possible, but that's going to be resolved soon.
This case is handled for continuous (distributed) evaluation differently. Instead of erroring out, we skip evaluation runs. That behavior is captured in the `test_skip_evaluation_due_to_ckpt` test.
PiperOrigin-RevId: 170919925
---
Commit 435b31b9f authored by Gunhan Gulsoy<gunan@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
BEGIN_PUBLIC
Automated g4 rollback of changelist 170892257
PiperOrigin-RevId: 171321707
END_PUBLIC
---
Commit 1e1b3d902 authored by Pete Warden<pete@petewarden.com>
Committed by gunan<gunan@google.com>:
Changed output directory for Pi CI build to fix permissions problem with nightlies (#13257)
* Fix for RTLD_GLOBAL breakage of Pi builds, and removed Eigen version change for Pi that's no longer needed
* Fixed Pi Zero OpenBLAS build problems and tidied up directories used
* More robust checks in Pi build script
* Changed output directory for Pi CI build to fix permissions problem
---
Commit fe3a2e65c authored by Yan Facai (???)<facai.yan@gmail.com>
Committed by drpngx<drpngx@users.noreply.github.com>:
check invalid string type for dest_nodes in extract_sub_graph (#13057)
* BUG: check str type
* TST: add unit test
* CLN: remove list check
* CLN: use warning
* CLN: 2 indent
* CLN: raise TypeError if not list
* CLN: check string only
---
Commit 225ab7629 authored by Jean Wanka<jm.wanka@gmail.com>
Committed by Jean Wanka<jm.wanka@gmail.com>:
Fix polynomial decay with cycle for global step=0
For polynomial decay with cycle=True the learning rate at
step 0 becomes NaN, because in the process of calculating it we
devide by 0. This change should fix it, by setting the multiplier
for the decay steps to one for global_step=0.
---
Commit 286f57061 authored by Bjarke Hammersholt Roune<broune@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Make Service::TransferToClient not attempt to manipulate the literal when the transfer failed, preventing a crash and allowing the caller to see the reason for the failed transfer.
PiperOrigin-RevId: 169770126
---
Commit e0501bc4d authored by Yong Tang<yong.tang.github@outlook.com>
Committed by Shanqing Cai<cais@google.com>:
Fix GRUBlockCell parameter naming inconsistency (#13153)
* Fix GRUBlockCell parameter naming inconsistency
This fix tries to fix the issue in 13137 where
parameter `cell_size` is used instead of `num_units`.
This is inconsistent with other RNN cells.
This fix adds support of `num_units` while at the same
time maintains backward compatiblility for `cell_size`.
This fix fixes 13137.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add `@deprecated_args` for 'cell_size' in `GRUBlockCell`
This commit adds `@deprecated_args` for 'cell_size' in `GRUBlockCell`
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Address review comment
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
---
Commit 02a2eba05 authored by Pete Warden<pete@petewarden.com>
Committed by gunan<gunan@google.com>:
Fix for RTLD_GLOBAL breakage of Pi builds, and removed Eigen version change that's no longer needed (#13251)
* Fix for RTLD_GLOBAL breakage of Pi builds, and removed Eigen version change for Pi that's no longer needed
* Fixed Pi Zero OpenBLAS build problems and tidied up directories used
* More robust checks in Pi build script
---
Commit 8ef722253 authored by Sanjoy Das<sanjoy@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Remove a redundant setName.
The EmitComputation should have emitted a function with the right name, so use a
CHECK instead.
PiperOrigin-RevId: 169764856
---
Commit 1b94147dc authored by Neal Wu<wun@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Fix broken GitHub links in tensorflow and tensorflow_models resulting from The Great Models Move (a.k.a. the research subfolder)
PiperOrigin-RevId: 169763373
---
Commit b1ada5f0c authored by Justine Tunney<jart@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Fix TensorBoard python -m invoke in docs
PiperOrigin-RevId: 169758752
---
Commit 2957cd894 authored by Mustafa Ispir<ispir@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
Local run option of estimator training.
PiperOrigin-RevId: 169756384
---
Commit 1dc2fe7ac authored by Gunhan Gulsoy<gunan@google.com>
Committed by TensorFlower Gardener<gardener@tensorflow.org>:
BEGIN_PUBLIC
Automated g4 rollback of changelist 166264198
PiperOrigin-RevId: 169998124
o Add new DataType kInt8
o Add new DataLayout kBatchDepthYX4 and new FilterLayout kOutputInputYX4, both
of which map to CUDNN_TENSOR_NCHW_VECT_C
o Change (Then|Do)TransformTensor to take input and output element types.
o Add new tensor format FORMAT_NCHW_VECT_C, and change the utility functions in
tensor_format.h to work with the new format.
PiperOrigin-RevId: 158806412
For now, use TF_CUDNN_USE_AUTOTUNE=1 to enable this feature. Once it is mature
enough, it will be turned on by default.
Support for the backward steps will be added later.
Change: 121769364
Change 109849574
Avoid some missing return warnings
Change 109837783
Add invalid aggregation to error message.
Change 109835474
Improves docstring of RegisterGradient decorator.
Fixes a typo (input -> output) and uses lowercase name for neg in the provided example.
Change 109834486
Update generated Op docs.
Change 109830497
Fix per_image_whitening to handle edge case by preventing the sqrt() of a negative number which is possible due to numerical floating point issues. Unit test added.
Fixes#411
Change 109824286
Change TensorBoard/TAG to 4
Change 109824197
Update tutorials and documentation usage of print to use print as function not statement.
This way you can copy+paste code in a python3 context and it will still work.
Change 109824020
Fix another case where TensorBoard discards values after a restart.
We also need to not discard on graph_def, since user code or SummaryWriter may add graph_defs at step 0 after every restart.
Change 109821924
Defines Templates for variable sharing.
A Template is a function that generates a sub-graph with the same variables each time it is called.
Two different templates defined with the same underlying function also return different variables.
Change 109815801
Don't instatiate the eigen expressions for additions and subtractions of
boolean since they won't be called. This reduces the size of the binary a bit.
Change 109795730
Allow casts to and from int8
Change 109791914
Python 3 fix: filter has no len
gradients.py calls len on the output of filter. A call to tuple is needed in
between.
Not sure why this wasn't caught when we ran the Python 3 tests. If I break it
for Python 2 several tests break.
Change 109757009
Fix minor grammatical errors in about.html
The missing article needs no justification, I think.
has -> have, because subjects are 'usability and functionality', not 'TensorFlow'.
and also -> and, because 'also' is superfluous in this use.
Change 109756627
TensorFlow: some doc updates to models/ files
Change 109743899
TensorFlow: remove one more clang warning (class / struct inconsistency).
Change 109741933
Document default for max_images in tf.image_summary
It used to say max_images=None which hid the C++ defalut of 3.
Now it says max_images=3.
Fixes https://github.com/tensorflow/tensorflow/issues/441.
It's unfortunate that an edit-distance-5 change produces such a large CL.
Change 109741569
Update generated Op docs.
Change 109739599
Renaming the Python variables in the layer weights of the fully connected
MNIST model so that the variable and the TensorFlow names are different. This
allows the documentation to be more explicit about the distinction between the
weights and biases of different layers. Also, the documentation gets to
describe the whether the TF name or the Python name is being used.
Base CL: 109851372
Change 109695551
Update FAQ
Change 109694725
Add a gradient for resize_bilinear op.
Change 109694505
Don't mention variables module in docs
variables.Variable should be tf.Variable.
Change 109658848
Adding an option to create a new thread-pool for each session.
Change 109640570
Take the snapshot of stream-executor.
+ Expose an interface for scratch space allocation in the interface.
Change 109638559
Let image_summary accept uint8 input
This allows users to do their own normalization / scaling if the default
(very weird) behavior of image_summary is undesired.
This required a slight tweak to fake_input.cc to make polymorphically typed
fake inputs infer if their type attr is not set but has a default.
Unfortunately, adding a second valid type to image_summary *disables* automatic
implicit conversion from np.float64 to tf.float32, so this change is slightly
backwards incompatible.
Change 109636969
Add serialization operations for SparseTensor.
Change 109636644
Update generated Op docs.
Change 109634899
TensorFlow: add a markdown file for producing release notes for our
releases. Seed with 0.5.0 with a boring but accurate description.
Change 109634502
Let histogram_summary take any realnumbertype
It used to take only floats, not it understands ints.
Change 109634434
TensorFlow: update locations where we mention python 3 support, update
them to current truth.
Change 109632108
Move HSV <> RGB conversions, grayscale conversions, and adjust_* ops back to tensorflow
- make GPU-capable version of RGBToHSV and HSVToRGB, allows only float input/output
- change docs to reflect new size constraints
- change HSV format to be [0,1] for all components
- add automatic dtype conversion for all adjust_* and grayscale conversion ops
- fix up docs
Change 109631077
Improve optimizer exceptions
1. grads_and_vars is now a tuple, so must be wrapped when passed to format.
2. Use '%r' instead of '%s' for dtype formatting
Base CL: 109697989
