experiments/STT-tensorflow
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[Java]: Remove references to decomissioned ci.tensorflow.org

Browse Source 
PiperOrigin-RevId: 206771402
This commit is contained in:
Asim Shankar 2018-07-31 09:09:51 -07:00 committed by TensorFlower Gardener
parent 5efec22b56
commit a7a9ea9764
3 changed files with 26 additions and 81 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
tensorflow/java/maven/README.md
View File

@ -151,16 +151,6 @@ conducted in a [Docker](https://www.docker.com) container.
7. Upon successful release, commit changes to all the `pom.xml` files 7. Upon successful release, commit changes to all the `pom.xml` files
(which should have the updated version number). (which should have the updated version number).
### Snapshots
If the `TF_VERSION` provided to the `release.sh` script ends in `-SNAPSHOT`,
then instead of using official release files, the nightly build artifacts from
https://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow/,
https://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow-windows/ and
https://ci.tensorflow.org/view/Nightly/job/nightly-android
will be used to upload to the Maven Central snapshots repository. (Note that
snapshots are only uploaded to Maven Central, not Bintray.)
### Skip deploying to a repository ### Skip deploying to a repository
Should you need, setting environment variables `DEPLOY_OSSRH=0` or Should you need, setting environment variables `DEPLOY_OSSRH=0` or
@ -173,12 +163,12 @@ cannot skip deploying to OSSRH for a `-SNAPSHOT` version.
This section provides some pointers around how artifacts are currently This section provides some pointers around how artifacts are currently
assembled. assembled.
All native and java code is first built and tested on All native and java code is first built and tested by the release process
a [Tensorflow Jenkins server](https://ci.tensorflow.org/) which run various which run various scripts under the [`tools/ci_build`](../../tools/ci_build/)
scripts under the [`tools/ci_build`](../../tools/ci_build/) directory. Of directory. Of particular interest may be
particular interest may be `tools/ci_build/builds/libtensorflow.sh` which `tools/ci_build/builds/libtensorflow.sh` which bundles Java-related build
bundles Java-related build sources and outputs into archives, and sources and outputs into archives, and `tools/ci_build/builds/android_full.sh`
`tools/ci_build/builds/android_full.sh` which produces an Android AAR package. which produces an Android AAR package.
Maven artifacts however are not created in Jenkins. Instead, artifacts are Maven artifacts however are not created in Jenkins. Instead, artifacts are
created and deployed externally on-demand, when a maintainer runs the created and deployed externally on-demand, when a maintainer runs the

68
tensorflow/java/maven/run_inside_container.sh
View File

@ -26,12 +26,6 @@ TF_ECOSYSTEM_URL="https://github.com/tensorflow/ecosystem.git"
DEPLOY_BINTRAY="${DEPLOY_BINTRAY:-true}" DEPLOY_BINTRAY="${DEPLOY_BINTRAY:-true}"
DEPLOY_OSSRH="${DEPLOY_OSSRH:-true}" DEPLOY_OSSRH="${DEPLOY_OSSRH:-true}"
IS_SNAPSHOT="false"
if [[ "${TF_VERSION}" == *"-SNAPSHOT" ]]; then
IS_SNAPSHOT="true"
# Bintray does not allow snapshots.
DEPLOY_BINTRAY="false"
fi
PROTOC_RELEASE_URL="https://github.com/google/protobuf/releases/download/v3.5.1/protoc-3.5.1-linux-x86_64.zip" PROTOC_RELEASE_URL="https://github.com/google/protobuf/releases/download/v3.5.1/protoc-3.5.1-linux-x86_64.zip"
if [[ "${DEPLOY_BINTRAY}" != "true" && "${DEPLOY_OSSRH}" != "true" ]]; then if [[ "${DEPLOY_BINTRAY}" != "true" && "${DEPLOY_OSSRH}" != "true" ]]; then
echo "Must deploy to at least one of Bintray or OSSRH" >&2 echo "Must deploy to at least one of Bintray or OSSRH" >&2
@ -69,11 +63,7 @@ mvn_property() {
} }
download_libtensorflow() { download_libtensorflow() {
if [[ "${IS_SNAPSHOT}" == "true" ]]; then URL="${RELEASE_URL_PREFIX}/libtensorflow-src-${TF_VERSION}.jar"
URL="http://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow/TYPE=cpu-slave/lastSuccessfulBuild/artifact/lib_package/libtensorflow-src.jar"
else
URL="${RELEASE_URL_PREFIX}/libtensorflow-src-${TF_VERSION}.jar"
fi
curl -L "${URL}" -o /tmp/src.jar curl -L "${URL}" -o /tmp/src.jar
cd "${DIR}/libtensorflow" cd "${DIR}/libtensorflow"
jar -xvf /tmp/src.jar jar -xvf /tmp/src.jar
@ -101,17 +91,9 @@ download_libtensorflow_jni() {
mkdir windows-x86_64 mkdir windows-x86_64
mkdir darwin-x86_64 mkdir darwin-x86_64
if [[ "${IS_SNAPSHOT}" == "true" ]]; then curl -L "${RELEASE_URL_PREFIX}/libtensorflow_jni-cpu-linux-x86_64-${TF_VERSION}.tar.gz" | tar -xvz -C linux-x86_64
# Nightly builds from http://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow/ curl -L "${RELEASE_URL_PREFIX}/libtensorflow_jni-cpu-darwin-x86_64-${TF_VERSION}.tar.gz" | tar -xvz -C darwin-x86_64
# and http://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow-windows/ curl -L "${RELEASE_URL_PREFIX}/libtensorflow_jni-cpu-windows-x86_64-${TF_VERSION}.zip" -o /tmp/windows.zip
curl -L "http://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow/TYPE=cpu-slave/lastSuccessfulBuild/artifact/lib_package/libtensorflow_jni-cpu-linux-x86_64.tar.gz" | tar -xvz -C linux-x86_64
curl -L "http://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow/TYPE=mac-slave/lastSuccessfulBuild/artifact/lib_package/libtensorflow_jni-cpu-darwin-x86_64.tar.gz" | tar -xvz -C darwin-x86_64
curl -L "http://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow-windows/lastSuccessfulBuild/artifact/lib_package/libtensorflow_jni-cpu-windows-x86_64.zip" -o /tmp/windows.zip
else
curl -L "${RELEASE_URL_PREFIX}/libtensorflow_jni-cpu-linux-x86_64-${TF_VERSION}.tar.gz" | tar -xvz -C linux-x86_64
curl -L "${RELEASE_URL_PREFIX}/libtensorflow_jni-cpu-darwin-x86_64-${TF_VERSION}.tar.gz" | tar -xvz -C darwin-x86_64
curl -L "${RELEASE_URL_PREFIX}/libtensorflow_jni-cpu-windows-x86_64-${TF_VERSION}.zip" -o /tmp/windows.zip
fi
unzip /tmp/windows.zip -d windows-x86_64 unzip /tmp/windows.zip -d windows-x86_64
rm -f /tmp/windows.zip rm -f /tmp/windows.zip
@ -129,13 +111,7 @@ download_libtensorflow_jni_gpu() {
mkdir linux-x86_64 mkdir linux-x86_64
if [[ "${IS_SNAPSHOT}" == "true" ]]; then curl -L "${RELEASE_URL_PREFIX}/libtensorflow_jni-gpu-linux-x86_64-${TF_VERSION}.tar.gz" | tar -xvz -C linux-x86_64
# Nightly builds from http://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow/
# and http://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow-windows/
curl -L "http://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow/TYPE=gpu-linux/lastSuccessfulBuild/artifact/lib_package/libtensorflow_jni-gpu-linux-x86_64.tar.gz" | tar -xvz -C linux-x86_64
else
curl -L "${RELEASE_URL_PREFIX}/libtensorflow_jni-gpu-linux-x86_64-${TF_VERSION}.tar.gz" | tar -xvz -C linux-x86_64
fi
# Updated timestamps seem to be required to get Maven to pick up the file. # Updated timestamps seem to be required to get Maven to pick up the file.
touch linux-x86_64/* touch linux-x86_64/*
@ -165,11 +141,7 @@ generate_java_protos() {
rm -f "/tmp/protoc.zip" rm -f "/tmp/protoc.zip"
# Download the release archive of TensorFlow protos. # Download the release archive of TensorFlow protos.
if [[ "${IS_SNAPSHOT}" == "true" ]]; then URL="${RELEASE_URL_PREFIX}/libtensorflow_proto-${TF_VERSION}.zip"
URL="http://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow/TYPE=cpu-slave/lastSuccessfulBuild/artifact/lib_package/libtensorflow_proto.zip"
else
URL="${RELEASE_URL_PREFIX}/libtensorflow_proto-${TF_VERSION}.zip"
fi
curl -L "${URL}" -o /tmp/libtensorflow_proto.zip curl -L "${URL}" -o /tmp/libtensorflow_proto.zip
mkdir -p "${DIR}/proto/tmp/src" mkdir -p "${DIR}/proto/tmp/src"
unzip -d "${DIR}/proto/tmp/src" "/tmp/libtensorflow_proto.zip" unzip -d "${DIR}/proto/tmp/src" "/tmp/libtensorflow_proto.zip"
@ -238,11 +210,7 @@ deploy_profile() {
# Determine the correct pom file property to use # Determine the correct pom file property to use
# for the repository url. # for the repository url.
local rtype local rtype
if [[ "${IS_SNAPSHOT}" == "true" ]]; then rtype='repository'
rtype='snapshotRepository'
else
rtype='repository'
fi
local url=$(mvn_property "${profile}" "project.distributionManagement.${rtype}.url") local url=$(mvn_property "${profile}" "project.distributionManagement.${rtype}.url")
local repositoryId=$(mvn_property "${profile}" "project.distributionManagement.${rtype}.id") local repositoryId=$(mvn_property "${profile}" "project.distributionManagement.${rtype}.id")
mvn gpg:sign-and-deploy-file \ mvn gpg:sign-and-deploy-file \
@ -300,17 +268,13 @@ mvn verify
deploy_artifacts deploy_artifacts
set +ex set +ex
if [[ "${IS_SNAPSHOT}" == "false" ]]; then echo "Uploaded to the staging repository"
echo "Uploaded to the staging repository" echo "After validating the release: "
echo "After validating the release: " if [[ "${DEPLOY_OSSRH}" == "true" ]]; then
if [[ "${DEPLOY_OSSRH}" == "true" ]]; then echo "* Login to https://oss.sonatype.org/#stagingRepositories"
echo "* Login to https://oss.sonatype.org/#stagingRepositories" echo "* Find the 'org.tensorflow' staging release and click either 'Release' to release or 'Drop' to abort"
echo "* Find the 'org.tensorflow' staging release and click either 'Release' to release or 'Drop' to abort" fi
fi if [[ "${DEPLOY_BINTRAY}" == "true" ]]; then
if [[ "${DEPLOY_BINTRAY}" == "true" ]]; then echo "* Login to https://bintray.com/google/tensorflow/tensorflow"
echo "* Login to https://bintray.com/google/tensorflow/tensorflow" echo "* Either 'Publish' unpublished items to release, or 'Discard' to abort"
echo "* Either 'Publish' unpublished items to release, or 'Discard' to abort"
fi
else
echo "Uploaded to the snapshot repository"
fi fi

17
tensorflow/java/maven/tensorflow-android/update.py
View File

@ -86,19 +86,10 @@ def read_template(path):
def main(): def main():
args = get_args() args = get_args()
# Artifacts are downloaded from the ci build. A SNAPSHOT release is release_prefix = 'https://storage.googleapis.com/tensorflow/libtensorflow'
# associated with artifacts from the last successful nightly build. Otherwise, info_url = '%s/android_buildinfo-%s.json' % (release_prefix, args.version)
# it comes from the officially blessed release artifacts. aar_url = '%s/tensorflow-%s.aar' % (release_prefix, args.version)
if args.version.endswith('SNAPSHOT'): build_type = 'release-android'
info_url = ('https://ci.tensorflow.org/view/Nightly/job/nightly-android'
'/lastSuccessfulBuild/api/json')
aar_url = None
build_type = 'nightly-android'
else:
release_prefix = 'https://storage.googleapis.com/tensorflow/libtensorflow'
info_url = '%s/android_buildinfo-%s.json' % (release_prefix, args.version)
aar_url = '%s/tensorflow-%s.aar' % (release_prefix, args.version)
build_type = 'release-android'
# Retrieve build information # Retrieve build information
build_info = get_json(info_url) build_info = get_json(info_url)