Commit Graph

1483 Commits

Author SHA1 Message Date
Frank Chen
23e387975e Rename tpu_load_library to tpu_api_dlsym_initializer.
This better reflects what the load library module does.

PiperOrigin-RevId: 317763973
Change-Id: I66169394c2f1d77b260323cb81d76f0f4e167579
2020-06-22 17:45:58 -07:00
Frank Chen
4a340a291f Add snappy compression support to RecordReader, RecordWriter, and tf.data snapshot
PiperOrigin-RevId: 317702626
Change-Id: Ic4c583387fb39646ac9e195ff96a3c5052da07c9
2020-06-22 12:46:40 -07:00
Frank Chen
27cb9aa834 Add SnappyInputStream implementation.
This is to support snappy compression/decompression on new version of snapshot dataset, as the dataset uses input streams instead.

PiperOrigin-RevId: 317435680
Change-Id: Ie57b43e73b6b7911398d883c3c5a0de72973288e
2020-06-19 22:53:29 -07:00
Frank Chen
cf00e559d7 Add TPU configuration ops to default TensorFlow build
PiperOrigin-RevId: 317133514
Change-Id: I33bc6d7fdbba5915bd0d1291d4e086139c07eb14
2020-06-18 10:52:49 -07:00
A. Unique TensorFlower
c34265b348 Integrate stackdriver support with tensorflow (roll forward cl/315012368)
PiperOrigin-RevId: 316580813
Change-Id: I93a66821fd13dc24a45c47f3e67638aed81d59be
2020-06-15 17:39:34 -07:00
Gunhan Gulsoy
2f402cbf23 Integrate stackdriver support with tensorflow
PiperOrigin-RevId: 315040782
Change-Id: I27eaadd8c1905c6265679e41d9811f004a9294e8
2020-06-05 20:38:38 -07:00
A. Unique TensorFlower
f5f25ec023 Integrate stackdriver support with tensorflow
PiperOrigin-RevId: 315012368
Change-Id: Ib41ec1cfe4920b8ebc22611dd6c86cdb4b43b018
2020-06-05 16:08:01 -07:00
Frank Chen
e56cf87b54 Adds necessary hooks to load a TPU-specific shared library.
PiperOrigin-RevId: 312601701
Change-Id: I1ae43d253d1734c30ffefe4d4062c82639d7a4d1
2020-05-20 19:34:16 -07:00
A. Unique TensorFlower
fd895bf2b9 [BUILD] Create a separate BUILD file for "tensorflow/core/protobuf"`.
This change leaves all existing targets in "tensorflow/core/BUILD" in place, with some becoming aliases. In future, we will remove aliases and point to the new locations.

PiperOrigin-RevId: 311194740
Change-Id: Id413277651b260641c1c2e06cb54d16629e6e662
2020-05-12 13:51:52 -07:00
A. Unique TensorFlower
6bb9ca398d Resolve trivial aliases for portable TensorFlow targets.
PiperOrigin-RevId: 311127183
Change-Id: I9011f48a3a753d0fdae5cff869a1b28ff1ccda3a
2020-05-12 08:14:33 -07:00
A. Unique TensorFlower
1514dd4025 Internal change only.
PiperOrigin-RevId: 310919530
Change-Id: I001957cc435c019c3611d728c8954cfe610ea6c8
2020-05-11 08:47:08 -07:00
A. Unique TensorFlower
fe3db61ad2 Merge portable full/lite proto libraries.
PiperOrigin-RevId: 310644217
Change-Id: I1589760a126a265f28b0be379b9b36ded3ddb3c9
2020-05-08 15:56:37 -07:00
A. Unique TensorFlower
3345a4083e Fold iOS-specific targets into the portable targets.
PiperOrigin-RevId: 310569018
Change-Id: Ic1bc71bbd6dd0f10c6d5442c5a30e587852a1eab
2020-05-08 09:20:28 -07:00
A. Unique TensorFlower
fedc6d951f Remove all uses of TENSORFLOW_LITE_PROTOS.
PiperOrigin-RevId: 310116123
Change-Id: I5fa28cc61644efc2fd202e80997a5c4c0a227572
2020-05-06 02:43:41 -07:00
Derek Murray
52542b8472 [Build cleanup] Remove "tensorflow/core/graph/graph_constructor.h".
This forwarding header was moved to "tensorflow/core/common_runtime/graph_constructor.h" and all known uses have been rewritten to use the version in "common_runtime".

PiperOrigin-RevId: 309767804
Change-Id: I92cc94f95ff02615a142bf19532f8677b3be36bc
2020-05-04 10:20:25 -07:00
A. Unique TensorFlower
f19618050b Remove tpu specific dependencies for Cast.
PiperOrigin-RevId: 309433861
Change-Id: I6607d5c98b0a1c8ce2389421eefb893fad20408e
2020-05-01 10:51:48 -07:00
A. Unique TensorFlower
b7c40bc3cc Add tf.sparse.bincount op.
PiperOrigin-RevId: 309334052
Change-Id: Ic00b2b66deb467dd901e0e7eaa7152e6d1dc18b0
2020-04-30 17:50:34 -07:00
Derek Murray
8ba3108452 [Build cleanup] Split "core_cpu_impl" into fine-grained targets (4/n).
This changes moves the remaining modules in "core/graph/..." that depend on modules in "core/common_runtime/..." to "core/common_runtime/", and creates separate fine-grained targets for each of them:

* "gradients.{h.cc}"
* "graph_def_builder_util.{h,cc}"
* "mkl_{layout,tfconversion}_pass.{h,cc}"
* "quantize_training.{h,cc}"

PiperOrigin-RevId: 308841156
Change-Id: I7774e2498f465290e7addd331625bd91d90acbac
2020-04-28 09:53:59 -07:00
Brian Zhao
375ec0ff00 Add a SavedModelAPI factory function to AbstractContextInterface, which allows a runtime-agnostic SavedModelAPI implementation. See RFC https://github.com/tensorflow/community/pull/207.
PiperOrigin-RevId: 308175037
Change-Id: Idd746fffbea3c17e0394cc359fb4920911fca687
2020-04-23 18:59:39 -07:00
Jared Duke
5a674e06a9 Fix msvc execution of TFLite C/Kernel tests
PiperOrigin-RevId: 307488714
Change-Id: Ia68e3479e5c8a8f6ac2222264638d2980e92127c
2020-04-20 15:33:40 -07:00
George Karpenkov
18809492b3 [NFC] Provide synchronous versions of CopyDeviceTensorToCPU and CopyCPUTensorToDevice
PiperOrigin-RevId: 307091950
Change-Id: I21eb8e775954d8227ae909180689e49b7a039149
2020-04-17 12:27:29 -07:00
Derek Murray
d4de6e7c21 [Build cleanup] Remove common_runtime dependency from "low_level_library_tests".
None of the tests in core/lib/ depend on libraries in core/common_runtime/.

PiperOrigin-RevId: 306710822
Change-Id: I55f18cee26fdd17cb35916bc47324d7733aebc80
2020-04-15 13:40:34 -07:00
Andrew Audibert
65e3fdb4ed [tf.data service] Package dataset ops that depend on grpc separately.
PiperOrigin-RevId: 306518696
Change-Id: I118f6599d93b149ae21f3eab63595008ded2692f
2020-04-14 14:51:55 -07:00
Gunhan Gulsoy
38ef9c25ae Remove non-existent dependency.
PiperOrigin-RevId: 306502691
Change-Id: I1cf0ffa12e6800cf691e5c189fd28806d8ed8cae
2020-04-14 13:31:30 -07:00
A. Unique TensorFlower
3acb43588d consolidate tf::profiler::ProfilerOptions with tf::ProfileOptions.
Remove ProfilerInterface::GetDeviceType();

PiperOrigin-RevId: 305506928
Change-Id: I18150a5af56d9a0950fa0f0fb5190150a495e806
2020-04-08 10:40:03 -07:00
Jiri Simsa
eabc157fd5 [tf.data] Adding a metric for bytes produced and consumed by individual transformations, refactoring infrastructure for recording tf.data metrics, and moving the metrics API and implementation from common_runtime to framework.
PiperOrigin-RevId: 305062865
Change-Id: I63911f00154baf36aa225f66dbef0843239b7392
2020-04-06 10:39:03 -07:00
TensorFlower Gardener
e530f61b9e Merge pull request from ROCmSoftwarePlatform:google_upstream_rocm_csb_fix_200311
PiperOrigin-RevId: 304743550
Change-Id: I0c14e5aec2f6f1cbc496f2bad0d6bd7a718c9241
2020-04-03 20:55:42 -07:00
Derek Murray
7f09d08f5d [BUILD] Clean up unused aliases left over the "common_runtime/BUILD" split.
PiperOrigin-RevId: 304650362
Change-Id: Ifc27d8d87d7c7319462235f661668467494dd252
2020-04-03 11:07:56 -07:00
Derek Murray
3f3bf39363 [BUILD] Create a separate BUILD file for "tensorflow/core/common_runtime"`.
Also create subsidiary BUILD files for "common_runtime/{gpu,sycl}".

This change leaves all existing targets in "tensorflow/core/BUILD" in place, with some becoming aliases. In future, we will split up the targets in "tensorflow/core/common_runtime/BUILD" to be more granular.

PiperOrigin-RevId: 304476929
Change-Id: Ida5348ce41b4d98337dd8a128bfcc44e20be9c3d
2020-04-02 14:47:41 -07:00
Gaurav Jain
6c7129eae0 Make core eager objects implement common interface
Abstract interface objects such as AbstractContextInterface,
AbstractOperationInterface & AbstractTensorHandleInterface were added to
decouple client code from the underlying implementation and allow for
easy switching to TFRT. Unfortunately the interface implementations had
to live in their own objects due to circular dependencies. This change
hopes to address most of them with the following changes & clean-ups:

* Move abstract interface definitions into separate build targets
* Make EagerContext implement AbstractContextInterface.
* Make EagerOperation implement AbstractOperationInterface.
* Make TensorHandle implement AbstractTensorHandleInterface.
* Use tensorflow::DataType instead of TF_DataType in interfaces.
* Replace unique_ptr with raw pointer and expose Release() function in
  abstract interfaces while making the destructor protected. This allows
  interface implementations to control their life-cycle, enabling
  EagerContext and TensorHandle which are reference counted to implement
  the interfaces directly.
* Make TensorHandle & EagerContext destructors private since the objects
  are reference counted.
* Move out interface methods from c_api.cc
* Use absl::Span instead of absl::FixedArray in operation interface.
* Make CustomDeviceAPI use TFE_DeleteTensorHandle instead of directly
  deleting the object.
* Remove unused TensorHandle::OnHostCPU method.
* Move remaining circular dependencies into core.cc.

PiperOrigin-RevId: 304269288
Change-Id: I2cced4a21ece708f413182b43f47e4891654901e
2020-04-01 15:03:12 -07:00
Derek Murray
bd530a65d5 [Executor] Split ExecutorState into PropagatorState and ExecutorState<PropagatorStateType>.
This change is part of an ongoing refactoring to simplify "executor.cc" and enable the substitution of more efficient implementations of `PropagateOutputs()`.

PiperOrigin-RevId: 304262448
Change-Id: I46a2d7fcdde89a71c502d272f35adfd34b0c4cab
2020-04-01 14:35:21 -07:00
Derek Murray
a1b5964cbb [Executor] Split out the immutable state of ExecutorImpl into ImmutableExecutorState.
In an ongoing effort to make "executor.cc" more readable, this change creates a new `ImmutableExecutorState` type containing most `ExecutorImpl` in "third_party/tensorflow/core/common_runtime/immutable_executor_state.h". It removes friend access from `ExecutorState` to the fields of `ExecutorImpl` and replaces it with public accessor methods for to the relevant fields.

PiperOrigin-RevId: 303886451
Change-Id: Ibea4e0d0b4fe88d10ca38705fbb2e6edc83f07ca
2020-03-30 21:22:22 -07:00
Derek Murray
41143f32b8 [Executor] Move GraphView and related types to a separate file.
In an ongoing effort to make "executor.cc" more readable, this change moves the following types to "third_party/tensorflow/core/common_runtime/graph_view.h":

* `ControlEdgeInfo`
* `EdgeInfo`
* `GraphView`
* `NodeItem`

PiperOrigin-RevId: 303438916
Change-Id: Ib49a18a70b5caa9338a18d179a6ceaad0dafe262
2020-03-27 17:32:53 -07:00
A. Unique TensorFlower
b97c90c87f Move third_party/tensorflow/core/protobuf/trace_events.proto.h to third_party/tensorflow/core/profiler/protobuf/trace_events.proto.h
PiperOrigin-RevId: 303188235
Change-Id: I06294ff7b4d2b38eb0b704c82dd53b1d02642c3f
2020-03-26 14:04:19 -07:00
A. Unique TensorFlower
1e07e069c1 Make portable_tensorflow_lib_lite work on iOS
PiperOrigin-RevId: 302747752
Change-Id: I120c0fc7bbdd9ac8828c43bd8df6ffb4b135f274
2020-03-24 14:34:10 -07:00
Yujing Zhang
8a53a3082b Support remote inputs passed as Tensors to EagerClusterFunctionLibraryRuntime::Run.
PiperOrigin-RevId: 302555474
Change-Id: I16d52b54748e255c2bf5ebcf522cb7a4eed639f0
2020-03-23 17:34:06 -07:00
Igor Ganichev
0ffd38260a Extend error message when super new GraphDef import fails
PiperOrigin-RevId: 302064508
Change-Id: Ibc3174157ebf91811628c6d49fb225d991d1f6c9
2020-03-20 11:26:58 -07:00
A. Unique TensorFlower
4c0d6b7d51 Changed TPU embedding load and retrieve ops to checked-in generated code.
PiperOrigin-RevId: 301887553
Change-Id: Ib6e042e73cd4a0214239175a4e86b090a0817f12
2020-03-19 13:52:03 -07:00
Gunhan Gulsoy
cfbf43aa1f Build more kernels into windows packages.
BUILD issues around these seem to be resolved now.

PiperOrigin-RevId: 301705450
Change-Id: I9ef82be3d3b3906dba520918aac4ad55a811c83a
2020-03-18 17:42:31 -07:00
Gunhan Gulsoy
89257f79df Build quantized ops into TF windows distribution.
PiperOrigin-RevId: 301699062
Change-Id: I21d5a5bcac2c53e90707fc45bbda90ef20abc1c7
2020-03-18 17:12:01 -07:00
Eugene Zhulenev
eaa1729a52 Force CPU placement for ops that has DT_VARIANT inputs with host-only underlying data type.
Fix for 

PiperOrigin-RevId: 301650148
Change-Id: I47fa9c1b0b7a7d56c5a519095687f36651892644
2020-03-18 13:09:23 -07:00
Derek Murray
749708ed0e Remove unused unique_tensor_references library.
This change also removes methods on `TensorReference` and `NodeExecStatsInterface` that are no longer used by any code; and fix some #includes and BUILD files that depended on indirect inclusion of "tensor_reference.h".

PiperOrigin-RevId: 300648897
Change-Id: I1f734b1de096f42158f7cd70ab9e27ad832db3f6
2020-03-12 16:35:29 -07:00
Deven Desai
3d3bf0031a Fix for broken ROCm CSB - 200309
This commit fixes regressions introduced in the ROCm CSB by th following commit :

df00d7ebbf

The above commit classifies puts "cudnn_rnn_kernels" under "if_cuda" in the tensorflow/core/BUILD file. That target is mis-named (should really be gpu_rnn_kernels) and is required by ROCm too. This commit simply changes the condition from "if_cuda" to "if_cuda_or_rocm"
2020-03-11 19:01:55 +00:00
A. Unique TensorFlower
0e4a3172a1 Add visibility to internal repos.
PiperOrigin-RevId: 300024750
Change-Id: Iffb70de4386f7773217e57dcd9ebee0c2a5846d2
2020-03-09 23:44:15 -07:00
Gunhan Gulsoy
60207a00d4 Disable failing placer_test on windows.
PiperOrigin-RevId: 299858784
Change-Id: I299bc7effb76eea0da27215b46a27ebb8ef6faec
2020-03-09 09:29:34 -07:00
Sanjoy Das
df00d7ebbf Move some GPU-only dependencies to under if_cuda_or_rocm
PiperOrigin-RevId: 299212692
Change-Id: Ia5db66af7acc199bc50cdee1869a23d707931c5b
2020-03-05 15:47:47 -08:00
Anna R
8f7e34982d Splitting out common_shape_fns.cc:
* Shape functions under shape_inference:: namespace stay in common_shape_fns.cc.
* Other functions are moved to kernel_shape_util.h.
Also, I removed common_shape_fns.h include in ops_util.h since it is not used in ops_util and only by files that include ops_util.h.

PiperOrigin-RevId: 298692539
Change-Id: I79df418a377d4eaed67aeec6f3736acb9d5b111d
2020-03-03 14:28:49 -08:00
A. Unique TensorFlower
5473ad460d Internal refactoring for metrics.
PiperOrigin-RevId: 297214293
Change-Id: I9ef1510677669bc9c21de5839fde7ed42421fb6d
2020-02-25 15:13:16 -08:00
Gunhan Gulsoy
18caa53810 Avoid linking core:lib and core:framework twice on windows.
And add yet another target to core/BUILD until we fully roll out
cc_shared_library :(
PiperOrigin-RevId: 297002883
Change-Id: Ieae6e732acfe6fffbbf0f7ff20189f1655661e9f
2020-02-24 16:56:27 -08:00
A. Unique TensorFlower
f396035891 Upgrade and rename external dependency grpc in workspace for bazel.
Fixes 

Downstream projects depending on TensorFlow: If bazel complains, please substitute `@zlib_archive` with `@zlib`, and `@grpc` with `@com_github_grpc_grpc` in WORKPLACE.

PiperOrigin-RevId: 295824868
Change-Id: If2259d59e9d82543369e5670916b1398374c9889
2020-02-18 14:40:26 -08:00