Bixia Zheng 14d708ab72 [TF:TRT] Handle out of GPU memory when creating TensorRT execution context. ... Previously, we use ICudaEngine::createExecutionContext to create a TensorRT execution context along with the GPU needed to execute the Cuda Engine. This API doesn't handle out of GPU memory properly, instead propagates an exception. This change uses ICudaEngine::createExecutionContextWithoutDeviceMemory to create a TensorRT execution context without any GPU memory, and let TF-TRT create the needed GPU memory. In order to keep track of such GPU memory, we wrap the TensorRT execution context and the associated GPU memory in a new class callsed ExecutionContext. PiperOrigin-RevId: 351895192 Change-Id: Ie01f0241578fadba8fad25bd110f937fd47082c8		2021-01-14 16:08:51 -08:00
..
common	[TF:TRT] Initialize TensorRT plugin registry before deserializing cuda engines.	2020-08-20 12:49:47 -07:00
convert	PR #46382: TF-TRT Test ConvertConcat in dynamic shape mode	2021-01-14 13:37:37 -08:00
kernels	[TF:TRT] Handle out of GPU memory when creating TensorRT execution context.	2021-01-14 16:08:51 -08:00
ops	[TF:TRT] Add support for per cluster maximum batch size.	2020-10-20 16:49:30 -07:00
plugin	[TF:TRT] Remove unused target plugin_cast.	2020-08-20 09:30:07 -07:00
segment	[TF:TRT] Add support for per cluster maximum batch size.	2020-10-20 16:49:30 -07:00
stub	Add TensorRT stub for 5.0 and 5.1.	2019-06-17 08:40:57 -07:00
utils	[TF:TRT] Handle out of GPU memory when creating TensorRT execution context.	2021-01-14 16:08:51 -08:00
BUILD	[TF:TRT] Handle out of GPU memory when creating TensorRT execution context.	2021-01-14 16:08:51 -08:00
tensorrt_test.cc	[TF:TRT] Add #if to logger_registry.h for consistency.	2020-06-22 19:19:16 -07:00