The version strings from cuda_configure.bzl are
cuda_version: x.y on Linux, 64_xy on Windows
cudnn_version: x on Linux, 64_x on Windows
The DLL names can be assembled from those, instead of passing them in from environment variables.
This hopefully fixes the Windows build.
PiperOrigin-RevId: 315212567
Change-Id: Ia9709297f8eff1baece1722e58e58f5bb469bbbf
This change is a second attempt at #38964, which was rolled back because it was fragile.
First, cuda_configure.bzl templates a file with data it already pulled from get_cuda_config. gen_build_info loads that file to provide package
build information within TensorFlow:
from tensorflow.python.platform import build_info
print(build_info.build_info)
{'cuda_version': '10.2', 'cudnn_version': '7', ... }
Also is exposed through tf.sysconfig.get_build_info(), a public API change.
setup.py pulls build_info into package metadata. The wheel's
long description ends with:
TensorFlow 2.2.0 for NVIDIA GPUs was built with these platform
and library versions:
- NVIDIA CUDA 10.2
- NVIDIA cuDNN 7
- NVIDIA CUDA Compute Capabilities compute_30, compute_70 (etc.)
I set one of the new CUDA Classifiers, and add metadata to the "platform" tag:
>>> import pkginfo
>>> a = pkginfo.Wheel('./tf_nightly_gpu-2.1.0-cp36-cp36m-linux_x86_64.whl')
>>> a.platforms
['cuda_version:10.2', 'cudnn_version:7', ...]
I'm not 100% confident this is the best way to accomplish this. It
still seems odd to import like this setup.py, even though it works, even in
an environment with TensorFlow installed. This method is much better than the old method as it uses data that was already gathered. It could be extended to gather tensorrt, nccl, etc. from other .bzl files, but I wanted to get feedback (and ensure this lands in 2.3) before designing something like that.
Currently tested only on Linux GPU (Remote Build) for Python 3.6. I'd
like to see more tests before merging.
The API is the same as the earlier change.
Resolves https://github.com/tensorflow/tensorflow/issues/38351.
PiperOrigin-RevId: 315018663
Change-Id: Idf68a8fe4d1585164d22b5870894c879537c280d
The local gen_build_info rule calls into find_cuda_config, which only works in the remote image.
This is additionally brittle: relying on TF_CUDA_VERSION being an action_env is poisoning our caches, and running find_cuda_conifg multiple times is bugprone.
I think the better way to do this is to put the information from the repo_rule into a file template as part of the repo rule configuration (cuda_configure.bzl). Then we can just include that file, instead of trying to do that as part of the action.
PiperOrigin-RevId: 311148754
Change-Id: I80daa8652a85b2a1897c15117e6422bfd21cee6a
Since this module now generates a dictionary to expose in tf.config, it
doesn't make much sense to store only certain values in the build_info
dictionary and others as module variables. This obsoletes a lot of code
in gen_build_info.py and I've removed it.
I also updated all the in-code references I've found to the build_info
module. I think this may break whomever used to be using the build_info
library, but since it wasn't part of the API, there was no guarantee
that it would continue to be available.
This change:
First exposes //third_party/gpus:find_cuda_config as a library.
Then, it extends gen_build_info.py with find_cuda_config to provide package
build information within TensorFlow's API. This is accessible as a
dictionary:
from tensorflow.python.platform import build_info
print(build_info.cuda_build_info)
{'cuda_version': '10.2', 'cudnn_version': '7', 'tensorrt_version': None, 'nccl_version': None}
Finally, setup.py pulls that into package metadata. The same wheel's
long description ends with:
TensorFlow 2.1.0 for NVIDIA GPUs was built with these platform
and library versions:
- NVIDIA CUDA 10.2
- NVIDIA cuDNN 7
- NVIDIA NCCL not enabled
- NVIDIA TensorRT not enabled
In lieu of NVIDIA CUDA classifiers [1], the same metadata is exposed in the
normally-unused "platform" tag:
>>> import pkginfo
>>> a = pkginfo.Wheel('./tf_nightly_gpu-2.1.0-cp36-cp36m-linux_x86_64.whl')
>>> a.platforms
['cuda_version:10.2', 'cudnn_version:7', 'tensorrt_version:None', 'nccl_version:None']
I'm not 100% confident this is the best way to accomplish this. It
seems odd to import like this setup.py, even though it works, even in
an environment with TensorFlow installed.
One caveat for RBE: the contents of genrules still run on the local
system, so I had to syncronize my local environment with the RBE
environment I used to build TensorFlow. I'm not sure if this is going to
require intervention on TensorFlow's current CI.
Currently tested only on Linux GPU (Remote Build) for Python 3.6. I'd
like to see more tests before merging.
[1]: (https://github.com/pypa/trove-classifiers/issues/25),
Add two repository rules:
- @local_execution_config_platform: local platform to allow selecting locally
executed tools on
- @local_execution_config_python: python configured for execution...
PiperOrigin-RevId: 307862682
Change-Id: Ie0320f2f137a40b418632989981c9dc072ef80e6
- @local_execution_config_platform: local platform to allow selecting locally
executed tools on
- @local_execution_config_python: python configured for execution on the local
machine during otherwise remote builds
Mark rules that are required to run locally to require our local platform.
This allows pyth...
PiperOrigin-RevId: 307771596
Change-Id: If1f0013ec88a35d507b2b622894208aab2416fe5
- @local_execution_config_platform: local platform to allow selecting locally
executed tools on
- @local_execution_config_python: python configured for execution on the local
machine during otherwise remote builds
Mark rules that are required to run locally to require our local platform.
This allows python paths to differ between the remote docker image and local
machine.
For example, the local machine might have python 3.7 installed in
/usr/bin/python3, while the remote docker should use a python installed
in /usr/local/bin/python3.8.
PiperOrigin-RevId: 307585019
Change-Id: I29313121beb967b77ae123e7d1b614c688cb40ca
- @local_execution_config_platform: local platform to allow selecting locally
executed tools on
- @local_execution_config_python: python configured for execution on the local
machine during otherwise remote builds
Mark rules that are required to run locally to require our local platform.
This allows python paths to differ between the remote docker image and local
machine.
For example, the local machine might have python 3.7 installed in
/usr/bin/python3, while the remote docker should use a python installed
in /usr/local/bin/python3.8.
PiperOrigin-RevId: 307558811
Change-Id: I0dc2d877a7c26b294bf2b569b4f121cf6506e7fc
This is mostly the result of an internal cleanup and formatting pass.
PiperOrigin-RevId: 286318018
Change-Id: I8f9e2f7519070035da73f9f24d2fc90864abc51b
The motivation for this script is to provide better tools for
diagnosing load-time errors (such as the ones that plague the Windows
build due to DLL issues). Note that the script is intended to be
self-contained, so that it is possible to import it without loading
the entire TensorFlow runtime.
This generated script currently contains a single symbol,
`is_cuda_build`, which records whether the build has GPU support or not.
PiperOrigin-RevId: 168471034
