experiments
/
STT-tensorflow
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Do not silently ignore ptxas compilation failures. 

Change the xla_gpu_unsafe_fallback_to_driver_on_ptxas_error default to false.

PiperOrigin-RevId: 311554370
Change-Id: I9a7f9ff114957998a84136e16333addf4a2cd354
This commit is contained in:
Thomas Joerg 2020-05-14 10:06:52 -07:00 committed by TensorFlower Gardener
parent ac4a8e17a5
commit 9dd3efb5aa
2 changed files with 7 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
tensorflow/compiler/xla/debug_options_flags.cc
View File

@ -66,7 +66,7 @@ DebugOptions DefaultDebugOptionsIgnoringFlags() {
opts.set_xla_cpu_enable_xprof_traceme(true);
// TODO(b/155295372): disable ptxas fallback by default.
opts.set_xla_gpu_unsafe_fallback_to_driver_on_ptxas_not_found(true);
opts.set_xla_gpu_unsafe_fallback_to_driver_on_ptxas_error(true);
opts.set_xla_gpu_unsafe_fallback_to_driver_on_ptxas_error(false);
return opts;
}

11
tensorflow/compiler/xla/service/gpu/nvptx_compiler.cc
View File

@ -416,11 +416,12 @@ std::vector<uint8> NVPTXCompiler::CompileGpuAsmOrGetCachedResult(
CHECK(hlo_module_config.debug_options()
.xla_gpu_unsafe_fallback_to_driver_on_ptxas_error())
<< "There was an error when trying to compile ptx into sass "
"code. If you want to try falling back to the GPU driver to "
"jit compile ptx, you can use the flag "
"--xla_gpu_unsafe_fallback_to_driver_on_ptxas_error."
" Use at your own risk though, it has known drawbacks like "
"increased memory consumption.";
"code. Up until May 14 2020, XLA silently ignored such "
"errors and fell back to the GPU driver. This is likely to "
"trigger subtle runtime issues and is hence discouraged. "
"If you want to temporarily restore this behavior use the "
"flag --xla_gpu_unsafe_fallback_to_driver_on_ptxas_error "
"and file a bug in b/components/366096.";
}
// We're going to use the driver to JIT our PTX->SASS, so warn if