STT-tensorflow/tensorflow/python/training/tracking
Daniel Ellis c29e9f25e7 Handle garbage collection race condition.
An exception is being thrown when objects that use `CapturableResourceDeleter` are garbage collected at the end of a program's life.  This can happen in very normal circumstances, such as when using `saved_model_cli` to inspect a model.

The cause of the exception appears to be a race condition with garbage collection between `CapturableResourceDeleter` and `ScopedTFFunction`. Both define a custom finalizer (`__del__`); `CaptureableResourceDeleter`'s finalizer ultimately calls a concrete function which calls an `_EagerDefinedFunction` which attempts to load and execute a `ScopedTFFunction`.

In the case of multiple objects in a reference cycle all going unreachable during the same garbage collection cycle, we get no guaranteed ordering for which of the objects will be collected first. In the case of the exception, `ScopedTFFunction` is collected first and its underlying function is deleted. Later, `CapturableResourceDeleter` is called, which fails, since the function it's trying to call is gone.

PiperOrigin-RevId: 358292164
Change-Id: I9162d5de622f5c1ec9b2954647b9958a7d3d87b6
2021-02-18 17:00:03 -08:00
..
base_test.py Move away from deprecated asserts 2020-06-30 16:10:22 -07:00
base.py Internal changes only. 2020-09-23 14:57:33 -07:00
benchmarks_test.py xrange() was removed from Python on 1/1/2020 2020-06-08 21:45:46 +02:00
BUILD [retry] Use same var key in _create_slots/get_slot in V1 optimizer 2021-01-28 10:48:00 -08:00
data_structures_test.py Fix incorrect usage of keras code in tf. 2020-11-12 15:34:01 -08:00
data_structures.py Support TF Modules inside Keras Layers and Models. 2020-10-30 12:27:12 -07:00
graph_view.py Rollback of: 2021-01-29 07:27:39 -08:00
layer_utils.py #PRIVATE_TF_API_USAGE_CLEANUP Remove the definition of gather_trainable_weights(), as there is no reference in the codebase. 2021-01-22 14:06:07 -08:00
python_state_test.py
python_state.py
tracking_test.py Move cached_per_instance to keras utils since its only used in Keras. 2020-08-14 10:14:53 -07:00
tracking.py Handle garbage collection race condition. 2021-02-18 17:00:03 -08:00
util_test.py Add option to construct tf.train.Checkpoint with a root object. 2020-08-14 12:19:56 -07:00
util_with_v1_optimizers_test.py Move away from deprecated asserts 2020-06-30 16:10:22 -07:00
util.py Update documentation to use proper tf.train.CheckpointOptions. 2021-02-02 10:50:53 -08:00