experiments/STT-tensorflow
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Remove -py3 docker images entirely

Browse Source 
This completes the plan announced in https://groups.google.com/a/tensorflow.org/d/msg/developers/5ZZG7M5E6FQ/zKv6uFn1EgAJ, removing all Python 2 considerations from our Dockerfiles. There are no longer any -py3 tags, as all the images are running Python 3.

Resolves #36971 and #37422. We're not able to dedicate the time to maintaining multiple versions of Python 3 at this time, unfortunately.

PiperOrigin-RevId: 304650599
Change-Id: Ib2b21e809f242fe914fb407381d8eaaa20f2ef99
This commit is contained in:
Austin Anderson 2020-04-03 10:58:21 -07:00 committed by TensorFlower Gardener
parent 7f09d08f5d
commit 9f7793af0b
26 changed files with 144 additions and 318 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
tensorflow/tools/dockerfiles/dockerfiles/cpu-jupyter.Dockerfile
View File

@ -25,25 +25,19 @@ FROM ubuntu:${UBUNTU_VERSION} as base
RUN apt-get update && apt-get install -y curl RUN apt-get update && apt-get install -y curl
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
# Options: # Options:
# tensorflow # tensorflow
@ -59,10 +53,9 @@ RUN ${PIP} install --no-cache-dir ${TF_PACKAGE}${TF_PACKAGE_VERSION:+==${TF_PACK
COPY bashrc /etc/bash.bashrc COPY bashrc /etc/bash.bashrc
RUN chmod a+rwx /etc/bash.bashrc RUN chmod a+rwx /etc/bash.bashrc
RUN ${PIP} install jupyter matplotlib RUN python3 -m pip install jupyter matplotlib
# Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422 # Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422
RUN if [[ "${USE_PYTHON_3_NOT_2}" == "1" ]]; then ${PIP} install ipykernel==5.1.1 nbformat==4.4.0; fi RUN python3 -m pip install jupyter_http_over_ws ipykernel==5.1.1 nbformat==4.4.0
RUN ${PIP} install jupyter_http_over_ws
RUN jupyter serverextension enable --py jupyter_http_over_ws RUN jupyter serverextension enable --py jupyter_http_over_ws
RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/ RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/
@ -82,6 +75,6 @@ RUN apt-get autoremove -y && apt-get remove -y wget
WORKDIR /tf WORKDIR /tf
EXPOSE 8888 EXPOSE 8888
RUN ${PYTHON} -m ipykernel.kernelspec RUN python3 -m ipykernel.kernelspec
CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"] CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"]

14
tensorflow/tools/dockerfiles/dockerfiles/cpu.Dockerfile
View File

@ -25,25 +25,19 @@ FROM ubuntu:${UBUNTU_VERSION} as base
RUN apt-get update && apt-get install -y curl RUN apt-get update && apt-get install -y curl
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
# Options: # Options:
# tensorflow # tensorflow

24
tensorflow/tools/dockerfiles/dockerfiles/devel-cpu-jupyter.Dockerfile
View File

@ -54,25 +54,19 @@ ARG CHECKOUT_TF_SRC=0
RUN chmod a+w /etc/passwd /etc/group RUN chmod a+w /etc/passwd /etc/group
RUN test "${CHECKOUT_TF_SRC}" -eq 1 && git clone https://github.com/tensorflow/tensorflow.git /tensorflow_src || true RUN test "${CHECKOUT_TF_SRC}" -eq 1 && git clone https://github.com/tensorflow/tensorflow.git /tensorflow_src || true
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
build-essential \ build-essential \
@ -80,7 +74,7 @@ RUN apt-get update && apt-get install -y \
git \ git \
wget \ wget \
openjdk-8-jdk \ openjdk-8-jdk \
${PYTHON}-dev \ python3-dev \
virtualenv \ virtualenv \
swig swig
@ -96,7 +90,6 @@ RUN ${PIP} --no-cache-dir install \
pandas \ pandas \
future \ future \
portpicker \ portpicker \
&& test "${USE_PYTHON_3_NOT_2}" -eq 1 && true || ${PIP} --no-cache-dir install \
enum34 enum34
# Install bazel # Install bazel
@ -111,10 +104,9 @@ RUN mkdir /bazel && \
COPY bashrc /etc/bash.bashrc COPY bashrc /etc/bash.bashrc
RUN chmod a+rwx /etc/bash.bashrc RUN chmod a+rwx /etc/bash.bashrc
RUN ${PIP} install jupyter matplotlib RUN python3 -m pip install jupyter matplotlib
# Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422 # Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422
RUN if [[ "${USE_PYTHON_3_NOT_2}" == "1" ]]; then ${PIP} install ipykernel==5.1.1 nbformat==4.4.0; fi RUN python3 -m pip install jupyter_http_over_ws ipykernel==5.1.1 nbformat==4.4.0
RUN ${PIP} install jupyter_http_over_ws
RUN jupyter serverextension enable --py jupyter_http_over_ws RUN jupyter serverextension enable --py jupyter_http_over_ws
RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/ RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/
@ -134,6 +126,6 @@ RUN apt-get autoremove -y && apt-get remove -y wget
WORKDIR /tf WORKDIR /tf
EXPOSE 8888 EXPOSE 8888
RUN ${PYTHON} -m ipykernel.kernelspec RUN python3 -m ipykernel.kernelspec
CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"] CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"]

17
tensorflow/tools/dockerfiles/dockerfiles/devel-cpu.Dockerfile
View File

@ -54,25 +54,19 @@ ARG CHECKOUT_TF_SRC=0
RUN chmod a+w /etc/passwd /etc/group RUN chmod a+w /etc/passwd /etc/group
RUN test "${CHECKOUT_TF_SRC}" -eq 1 && git clone https://github.com/tensorflow/tensorflow.git /tensorflow_src || true RUN test "${CHECKOUT_TF_SRC}" -eq 1 && git clone https://github.com/tensorflow/tensorflow.git /tensorflow_src || true
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
build-essential \ build-essential \
@ -80,7 +74,7 @@ RUN apt-get update && apt-get install -y \
git \ git \
wget \ wget \
openjdk-8-jdk \ openjdk-8-jdk \
${PYTHON}-dev \ python3-dev \
virtualenv \ virtualenv \
swig swig
@ -96,7 +90,6 @@ RUN ${PIP} --no-cache-dir install \
pandas \ pandas \
future \ future \
portpicker \ portpicker \
&& test "${USE_PYTHON_3_NOT_2}" -eq 1 && true || ${PIP} --no-cache-dir install \
enum34 enum34
# Install bazel # Install bazel

24
tensorflow/tools/dockerfiles/dockerfiles/devel-gpu-jupyter.Dockerfile
View File

@ -96,25 +96,19 @@ RUN ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/lib
&& echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \ && echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \
&& ldconfig && ldconfig
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
build-essential \ build-essential \
@ -122,7 +116,7 @@ RUN apt-get update && apt-get install -y \
git \ git \
wget \ wget \
openjdk-8-jdk \ openjdk-8-jdk \
${PYTHON}-dev \ python3-dev \
virtualenv \ virtualenv \
swig swig
@ -138,7 +132,6 @@ RUN ${PIP} --no-cache-dir install \
pandas \ pandas \
future \ future \
portpicker \ portpicker \
&& test "${USE_PYTHON_3_NOT_2}" -eq 1 && true || ${PIP} --no-cache-dir install \
enum34 enum34
# Install bazel # Install bazel
@ -153,10 +146,9 @@ RUN mkdir /bazel && \
COPY bashrc /etc/bash.bashrc COPY bashrc /etc/bash.bashrc
RUN chmod a+rwx /etc/bash.bashrc RUN chmod a+rwx /etc/bash.bashrc
RUN ${PIP} install jupyter matplotlib RUN python3 -m pip install jupyter matplotlib
# Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422 # Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422
RUN if [[ "${USE_PYTHON_3_NOT_2}" == "1" ]]; then ${PIP} install ipykernel==5.1.1 nbformat==4.4.0; fi RUN python3 -m pip install jupyter_http_over_ws ipykernel==5.1.1 nbformat==4.4.0
RUN ${PIP} install jupyter_http_over_ws
RUN jupyter serverextension enable --py jupyter_http_over_ws RUN jupyter serverextension enable --py jupyter_http_over_ws
RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/ RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/
@ -176,6 +168,6 @@ RUN apt-get autoremove -y && apt-get remove -y wget
WORKDIR /tf WORKDIR /tf
EXPOSE 8888 EXPOSE 8888
RUN ${PYTHON} -m ipykernel.kernelspec RUN python3 -m ipykernel.kernelspec
CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"] CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"]

17
tensorflow/tools/dockerfiles/dockerfiles/devel-gpu.Dockerfile
View File

@ -96,25 +96,19 @@ RUN ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/lib
&& echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \ && echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \
&& ldconfig && ldconfig
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
build-essential \ build-essential \
@ -122,7 +116,7 @@ RUN apt-get update && apt-get install -y \
git \ git \
wget \ wget \
openjdk-8-jdk \ openjdk-8-jdk \
${PYTHON}-dev \ python3-dev \
virtualenv \ virtualenv \
swig swig
@ -138,7 +132,6 @@ RUN ${PIP} --no-cache-dir install \
pandas \ pandas \
future \ future \
portpicker \ portpicker \
&& test "${USE_PYTHON_3_NOT_2}" -eq 1 && true || ${PIP} --no-cache-dir install \
enum34 enum34
# Install bazel # Install bazel

21
tensorflow/tools/dockerfiles/dockerfiles/gpu-jupyter.Dockerfile
View File

@ -74,25 +74,19 @@ RUN ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/lib
&& echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \ && echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \
&& ldconfig && ldconfig
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
# Options: # Options:
# tensorflow # tensorflow
@ -108,10 +102,9 @@ RUN ${PIP} install --no-cache-dir ${TF_PACKAGE}${TF_PACKAGE_VERSION:+==${TF_PACK
COPY bashrc /etc/bash.bashrc COPY bashrc /etc/bash.bashrc
RUN chmod a+rwx /etc/bash.bashrc RUN chmod a+rwx /etc/bash.bashrc
RUN ${PIP} install jupyter matplotlib RUN python3 -m pip install jupyter matplotlib
# Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422 # Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422
RUN if [[ "${USE_PYTHON_3_NOT_2}" == "1" ]]; then ${PIP} install ipykernel==5.1.1 nbformat==4.4.0; fi RUN python3 -m pip install jupyter_http_over_ws ipykernel==5.1.1 nbformat==4.4.0
RUN ${PIP} install jupyter_http_over_ws
RUN jupyter serverextension enable --py jupyter_http_over_ws RUN jupyter serverextension enable --py jupyter_http_over_ws
RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/ RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/
@ -131,6 +124,6 @@ RUN apt-get autoremove -y && apt-get remove -y wget
WORKDIR /tf WORKDIR /tf
EXPOSE 8888 EXPOSE 8888
RUN ${PYTHON} -m ipykernel.kernelspec RUN python3 -m ipykernel.kernelspec
CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"] CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"]

14
tensorflow/tools/dockerfiles/dockerfiles/gpu.Dockerfile
View File

@ -74,25 +74,19 @@ RUN ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/lib
&& echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \ && echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \
&& ldconfig && ldconfig
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
# Options: # Options:
# tensorflow # tensorflow

26
tensorflow/tools/dockerfiles/dockerfiles/mkl_horovod/devel-horovod-jupyter.Dockerfile
View File

@ -54,25 +54,19 @@ ARG CHECKOUT_TF_SRC=0
RUN chmod a+w /etc/passwd /etc/group RUN chmod a+w /etc/passwd /etc/group
RUN test "${CHECKOUT_TF_SRC}" -eq 1 && git clone https://github.com/tensorflow/tensorflow.git /tensorflow_src || true RUN test "${CHECKOUT_TF_SRC}" -eq 1 && git clone https://github.com/tensorflow/tensorflow.git /tensorflow_src || true
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
build-essential \ build-essential \
@ -80,7 +74,7 @@ RUN apt-get update && apt-get install -y \
git \ git \
wget \ wget \
openjdk-8-jdk \ openjdk-8-jdk \
${PYTHON}-dev \ python3-dev \
virtualenv \ virtualenv \
swig swig
@ -96,11 +90,10 @@ RUN ${PIP} --no-cache-dir install \
pandas \ pandas \
future \ future \
portpicker \ portpicker \
&& test "${USE_PYTHON_3_NOT_2}" -eq 1 && true || ${PIP} --no-cache-dir install \
enum34 enum34
# Install bazel # Install bazel
ARG BAZEL_VERSION=1.2.1 ARG BAZEL_VERSION=2.0.0
RUN mkdir /bazel && \ RUN mkdir /bazel && \
wget -O /bazel/installer.sh "https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh" && \ wget -O /bazel/installer.sh "https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh" && \
wget -O /bazel/LICENSE.txt "https://raw.githubusercontent.com/bazelbuild/bazel/master/LICENSE" && \ wget -O /bazel/LICENSE.txt "https://raw.githubusercontent.com/bazelbuild/bazel/master/LICENSE" && \
@ -163,10 +156,9 @@ RUN test "${CHECKOUT_HOROVOD_SRC}" -eq 1 && git clone --recursive https://github
COPY bashrc /etc/bash.bashrc COPY bashrc /etc/bash.bashrc
RUN chmod a+rwx /etc/bash.bashrc RUN chmod a+rwx /etc/bash.bashrc
RUN ${PIP} install jupyter matplotlib RUN python3 -m pip install jupyter matplotlib
# Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422 # Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422
RUN if [[ "${USE_PYTHON_3_NOT_2}" == "1" ]]; then ${PIP} install ipykernel==5.1.1 nbformat==4.4.0; fi RUN python3 -m pip install jupyter_http_over_ws ipykernel==5.1.1 nbformat==4.4.0
RUN ${PIP} install jupyter_http_over_ws
RUN jupyter serverextension enable --py jupyter_http_over_ws RUN jupyter serverextension enable --py jupyter_http_over_ws
RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/ RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/
@ -186,6 +178,6 @@ RUN apt-get autoremove -y && apt-get remove -y wget
WORKDIR /tf WORKDIR /tf
EXPOSE 8888 EXPOSE 8888
RUN ${PYTHON} -m ipykernel.kernelspec RUN python3 -m ipykernel.kernelspec
CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"] CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"]

19
tensorflow/tools/dockerfiles/dockerfiles/mkl_horovod/devel-horovod.Dockerfile
View File

@ -54,25 +54,19 @@ ARG CHECKOUT_TF_SRC=0
RUN chmod a+w /etc/passwd /etc/group RUN chmod a+w /etc/passwd /etc/group
RUN test "${CHECKOUT_TF_SRC}" -eq 1 && git clone https://github.com/tensorflow/tensorflow.git /tensorflow_src || true RUN test "${CHECKOUT_TF_SRC}" -eq 1 && git clone https://github.com/tensorflow/tensorflow.git /tensorflow_src || true
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
build-essential \ build-essential \
@ -80,7 +74,7 @@ RUN apt-get update && apt-get install -y \
git \ git \
wget \ wget \
openjdk-8-jdk \ openjdk-8-jdk \
${PYTHON}-dev \ python3-dev \
virtualenv \ virtualenv \
swig swig
@ -96,11 +90,10 @@ RUN ${PIP} --no-cache-dir install \
pandas \ pandas \
future \ future \
portpicker \ portpicker \
&& test "${USE_PYTHON_3_NOT_2}" -eq 1 && true || ${PIP} --no-cache-dir install \
enum34 enum34
# Install bazel # Install bazel
ARG BAZEL_VERSION=1.2.1 ARG BAZEL_VERSION=2.0.0
RUN mkdir /bazel && \ RUN mkdir /bazel && \
wget -O /bazel/installer.sh "https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh" && \ wget -O /bazel/installer.sh "https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh" && \
wget -O /bazel/LICENSE.txt "https://raw.githubusercontent.com/bazelbuild/bazel/master/LICENSE" && \ wget -O /bazel/LICENSE.txt "https://raw.githubusercontent.com/bazelbuild/bazel/master/LICENSE" && \

21
tensorflow/tools/dockerfiles/dockerfiles/mkl_horovod/horovod-jupyter.Dockerfile
View File

@ -25,25 +25,19 @@ FROM ubuntu:${UBUNTU_VERSION} as base
RUN apt-get update && apt-get install -y curl RUN apt-get update && apt-get install -y curl
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
# Options: # Options:
# tensorflow # tensorflow
@ -111,10 +105,9 @@ RUN ${PIP} install --no-cache-dir horovod==${HOROVOD_VERSION}
COPY bashrc /etc/bash.bashrc COPY bashrc /etc/bash.bashrc
RUN chmod a+rwx /etc/bash.bashrc RUN chmod a+rwx /etc/bash.bashrc
RUN ${PIP} install jupyter matplotlib RUN python3 -m pip install jupyter matplotlib
# Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422 # Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422
RUN if [[ "${USE_PYTHON_3_NOT_2}" == "1" ]]; then ${PIP} install ipykernel==5.1.1 nbformat==4.4.0; fi RUN python3 -m pip install jupyter_http_over_ws ipykernel==5.1.1 nbformat==4.4.0
RUN ${PIP} install jupyter_http_over_ws
RUN jupyter serverextension enable --py jupyter_http_over_ws RUN jupyter serverextension enable --py jupyter_http_over_ws
RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/ RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/
@ -134,6 +127,6 @@ RUN apt-get autoremove -y && apt-get remove -y wget
WORKDIR /tf WORKDIR /tf
EXPOSE 8888 EXPOSE 8888
RUN ${PYTHON} -m ipykernel.kernelspec RUN python3 -m ipykernel.kernelspec
CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"] CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"]

14
tensorflow/tools/dockerfiles/dockerfiles/mkl_horovod/horovod.Dockerfile
View File

@ -25,25 +25,19 @@ FROM ubuntu:${UBUNTU_VERSION} as base
RUN apt-get update && apt-get install -y curl RUN apt-get update && apt-get install -y curl
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
# Options: # Options:
# tensorflow # tensorflow

25
tensorflow/tools/dockerfiles/dockerfiles/ppc64le/cpu-ppc64le-jupyter.Dockerfile
View File

@ -25,25 +25,19 @@ FROM ubuntu:${UBUNTU_VERSION} as base
RUN apt-get update && apt-get install -y curl RUN apt-get update && apt-get install -y curl
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
# Options: # Options:
# tensorflow # tensorflow
@ -68,8 +62,8 @@ RUN if [ ${TF_PACKAGE} = tensorflow-gpu ]; then \
elif [ ${TF_PACKAGE} = tf-nightly ]; then \ elif [ ${TF_PACKAGE} = tf-nightly ]; then \
BASE=https://powerci.osuosl.org/job/TensorFlow_PPC64LE_CPU_Nightly_Artifact/lastSuccessfulBuild/; \ BASE=https://powerci.osuosl.org/job/TensorFlow_PPC64LE_CPU_Nightly_Artifact/lastSuccessfulBuild/; \
fi; \ fi; \
MAJOR=`${PYTHON} -c 'import sys; print(sys.version_info[0])'`; \ MAJOR=`python3 -c 'import sys; print(sys.version_info[0])'`; \
MINOR=`${PYTHON} -c 'import sys; print(sys.version_info[1])'`; \ MINOR=`python3 -c 'import sys; print(sys.version_info[1])'`; \
PACKAGE=$(wget -qO- ${BASE}"api/xml?xpath=//fileName&wrapper=artifacts" | grep -o "[^<>]*cp${MAJOR}${MINOR}[^<>]*.whl"); \ PACKAGE=$(wget -qO- ${BASE}"api/xml?xpath=//fileName&wrapper=artifacts" | grep -o "[^<>]*cp${MAJOR}${MINOR}[^<>]*.whl"); \
wget ${BASE}"artifact/tensorflow_pkg/"${PACKAGE}; \ wget ${BASE}"artifact/tensorflow_pkg/"${PACKAGE}; \
${PIP} install ${PACKAGE} ${PIP} install ${PACKAGE}
@ -77,10 +71,9 @@ RUN if [ ${TF_PACKAGE} = tensorflow-gpu ]; then \
COPY bashrc /etc/bash.bashrc COPY bashrc /etc/bash.bashrc
RUN chmod a+rwx /etc/bash.bashrc RUN chmod a+rwx /etc/bash.bashrc
RUN ${PIP} install jupyter matplotlib RUN python3 -m pip install jupyter matplotlib
# Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422 # Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422
RUN if [[ "${USE_PYTHON_3_NOT_2}" == "1" ]]; then ${PIP} install ipykernel==5.1.1 nbformat==4.4.0; fi RUN python3 -m pip install jupyter_http_over_ws ipykernel==5.1.1 nbformat==4.4.0
RUN ${PIP} install jupyter_http_over_ws
RUN jupyter serverextension enable --py jupyter_http_over_ws RUN jupyter serverextension enable --py jupyter_http_over_ws
RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/ RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/
@ -100,6 +93,6 @@ RUN apt-get autoremove -y && apt-get remove -y wget
WORKDIR /tf WORKDIR /tf
EXPOSE 8888 EXPOSE 8888
RUN ${PYTHON} -m ipykernel.kernelspec RUN python3 -m ipykernel.kernelspec
CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"] CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"]

18
tensorflow/tools/dockerfiles/dockerfiles/ppc64le/cpu-ppc64le.Dockerfile
View File

@ -25,25 +25,19 @@ FROM ubuntu:${UBUNTU_VERSION} as base
RUN apt-get update && apt-get install -y curl RUN apt-get update && apt-get install -y curl
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
# Options: # Options:
# tensorflow # tensorflow
@ -68,8 +62,8 @@ RUN if [ ${TF_PACKAGE} = tensorflow-gpu ]; then \
elif [ ${TF_PACKAGE} = tf-nightly ]; then \ elif [ ${TF_PACKAGE} = tf-nightly ]; then \
BASE=https://powerci.osuosl.org/job/TensorFlow_PPC64LE_CPU_Nightly_Artifact/lastSuccessfulBuild/; \ BASE=https://powerci.osuosl.org/job/TensorFlow_PPC64LE_CPU_Nightly_Artifact/lastSuccessfulBuild/; \
fi; \ fi; \
MAJOR=`${PYTHON} -c 'import sys; print(sys.version_info[0])'`; \ MAJOR=`python3 -c 'import sys; print(sys.version_info[0])'`; \
MINOR=`${PYTHON} -c 'import sys; print(sys.version_info[1])'`; \ MINOR=`python3 -c 'import sys; print(sys.version_info[1])'`; \
PACKAGE=$(wget -qO- ${BASE}"api/xml?xpath=//fileName&wrapper=artifacts" | grep -o "[^<>]*cp${MAJOR}${MINOR}[^<>]*.whl"); \ PACKAGE=$(wget -qO- ${BASE}"api/xml?xpath=//fileName&wrapper=artifacts" | grep -o "[^<>]*cp${MAJOR}${MINOR}[^<>]*.whl"); \
wget ${BASE}"artifact/tensorflow_pkg/"${PACKAGE}; \ wget ${BASE}"artifact/tensorflow_pkg/"${PACKAGE}; \
${PIP} install ${PACKAGE} ${PIP} install ${PACKAGE}

24
tensorflow/tools/dockerfiles/dockerfiles/ppc64le/devel-cpu-ppc64le-jupyter.Dockerfile
View File

@ -54,32 +54,26 @@ ARG CHECKOUT_TF_SRC=0
RUN chmod a+w /etc/passwd /etc/group RUN chmod a+w /etc/passwd /etc/group
RUN test "${CHECKOUT_TF_SRC}" -eq 1 && git clone https://github.com/tensorflow/tensorflow.git /tensorflow_src || true RUN test "${CHECKOUT_TF_SRC}" -eq 1 && git clone https://github.com/tensorflow/tensorflow.git /tensorflow_src || true
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
build-essential \ build-essential \
curl \ curl \
git \ git \
openjdk-8-jdk \ openjdk-8-jdk \
${PYTHON}-dev \ python3-dev \
virtualenv \ virtualenv \
swig swig
@ -94,7 +88,6 @@ RUN ${PIP} --no-cache-dir install \
sklearn \ sklearn \
pandas \ pandas \
portpicker \ portpicker \
&& test "${USE_PYTHON_3_NOT_2}" -eq 1 && true || ${PIP} --no-cache-dir install \
enum34 enum34
# Build and install bazel # Build and install bazel
@ -112,10 +105,9 @@ RUN mkdir /bazel && \
COPY bashrc /etc/bash.bashrc COPY bashrc /etc/bash.bashrc
RUN chmod a+rwx /etc/bash.bashrc RUN chmod a+rwx /etc/bash.bashrc
RUN ${PIP} install jupyter matplotlib RUN python3 -m pip install jupyter matplotlib
# Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422 # Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422
RUN if [[ "${USE_PYTHON_3_NOT_2}" == "1" ]]; then ${PIP} install ipykernel==5.1.1 nbformat==4.4.0; fi RUN python3 -m pip install jupyter_http_over_ws ipykernel==5.1.1 nbformat==4.4.0
RUN ${PIP} install jupyter_http_over_ws
RUN jupyter serverextension enable --py jupyter_http_over_ws RUN jupyter serverextension enable --py jupyter_http_over_ws
RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/ RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/
@ -135,6 +127,6 @@ RUN apt-get autoremove -y && apt-get remove -y wget
WORKDIR /tf WORKDIR /tf
EXPOSE 8888 EXPOSE 8888
RUN ${PYTHON} -m ipykernel.kernelspec RUN python3 -m ipykernel.kernelspec
CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"] CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"]

17
tensorflow/tools/dockerfiles/dockerfiles/ppc64le/devel-cpu-ppc64le.Dockerfile
View File

@ -54,32 +54,26 @@ ARG CHECKOUT_TF_SRC=0
RUN chmod a+w /etc/passwd /etc/group RUN chmod a+w /etc/passwd /etc/group
RUN test "${CHECKOUT_TF_SRC}" -eq 1 && git clone https://github.com/tensorflow/tensorflow.git /tensorflow_src || true RUN test "${CHECKOUT_TF_SRC}" -eq 1 && git clone https://github.com/tensorflow/tensorflow.git /tensorflow_src || true
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
build-essential \ build-essential \
curl \ curl \
git \ git \
openjdk-8-jdk \ openjdk-8-jdk \
${PYTHON}-dev \ python3-dev \
virtualenv \ virtualenv \
swig swig
@ -94,7 +88,6 @@ RUN ${PIP} --no-cache-dir install \
sklearn \ sklearn \
pandas \ pandas \
portpicker \ portpicker \
&& test "${USE_PYTHON_3_NOT_2}" -eq 1 && true || ${PIP} --no-cache-dir install \
enum34 enum34
# Build and install bazel # Build and install bazel

24
tensorflow/tools/dockerfiles/dockerfiles/ppc64le/devel-gpu-ppc64le-jupyter.Dockerfile
View File

@ -96,32 +96,26 @@ RUN ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/lib
&& echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \ && echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \
&& ldconfig && ldconfig
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
build-essential \ build-essential \
curl \ curl \
git \ git \
openjdk-8-jdk \ openjdk-8-jdk \
${PYTHON}-dev \ python3-dev \
virtualenv \ virtualenv \
swig swig
@ -136,7 +130,6 @@ RUN ${PIP} --no-cache-dir install \
sklearn \ sklearn \
pandas \ pandas \
portpicker \ portpicker \
&& test "${USE_PYTHON_3_NOT_2}" -eq 1 && true || ${PIP} --no-cache-dir install \
enum34 enum34
# Build and install bazel # Build and install bazel
@ -154,10 +147,9 @@ RUN mkdir /bazel && \
COPY bashrc /etc/bash.bashrc COPY bashrc /etc/bash.bashrc
RUN chmod a+rwx /etc/bash.bashrc RUN chmod a+rwx /etc/bash.bashrc
RUN ${PIP} install jupyter matplotlib RUN python3 -m pip install jupyter matplotlib
# Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422 # Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422
RUN if [[ "${USE_PYTHON_3_NOT_2}" == "1" ]]; then ${PIP} install ipykernel==5.1.1 nbformat==4.4.0; fi RUN python3 -m pip install jupyter_http_over_ws ipykernel==5.1.1 nbformat==4.4.0
RUN ${PIP} install jupyter_http_over_ws
RUN jupyter serverextension enable --py jupyter_http_over_ws RUN jupyter serverextension enable --py jupyter_http_over_ws
RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/ RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/
@ -177,6 +169,6 @@ RUN apt-get autoremove -y && apt-get remove -y wget
WORKDIR /tf WORKDIR /tf
EXPOSE 8888 EXPOSE 8888
RUN ${PYTHON} -m ipykernel.kernelspec RUN python3 -m ipykernel.kernelspec
CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"] CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"]

17
tensorflow/tools/dockerfiles/dockerfiles/ppc64le/devel-gpu-ppc64le.Dockerfile
View File

@ -96,32 +96,26 @@ RUN ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/lib
&& echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \ && echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \
&& ldconfig && ldconfig
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
build-essential \ build-essential \
curl \ curl \
git \ git \
openjdk-8-jdk \ openjdk-8-jdk \
${PYTHON}-dev \ python3-dev \
virtualenv \ virtualenv \
swig swig
@ -136,7 +130,6 @@ RUN ${PIP} --no-cache-dir install \
sklearn \ sklearn \
pandas \ pandas \
portpicker \ portpicker \
&& test "${USE_PYTHON_3_NOT_2}" -eq 1 && true || ${PIP} --no-cache-dir install \
enum34 enum34
# Build and install bazel # Build and install bazel

25
tensorflow/tools/dockerfiles/dockerfiles/ppc64le/gpu-ppc64le-jupyter.Dockerfile
View File

@ -74,25 +74,19 @@ RUN ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/lib
&& echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \ && echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \
&& ldconfig && ldconfig
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
# Options: # Options:
# tensorflow # tensorflow
@ -117,8 +111,8 @@ RUN if [ ${TF_PACKAGE} = tensorflow-gpu ]; then \
elif [ ${TF_PACKAGE} = tf-nightly ]; then \ elif [ ${TF_PACKAGE} = tf-nightly ]; then \
BASE=https://powerci.osuosl.org/job/TensorFlow_PPC64LE_CPU_Nightly_Artifact/lastSuccessfulBuild/; \ BASE=https://powerci.osuosl.org/job/TensorFlow_PPC64LE_CPU_Nightly_Artifact/lastSuccessfulBuild/; \
fi; \ fi; \
MAJOR=`${PYTHON} -c 'import sys; print(sys.version_info[0])'`; \ MAJOR=`python3 -c 'import sys; print(sys.version_info[0])'`; \
MINOR=`${PYTHON} -c 'import sys; print(sys.version_info[1])'`; \ MINOR=`python3 -c 'import sys; print(sys.version_info[1])'`; \
PACKAGE=$(wget -qO- ${BASE}"api/xml?xpath=//fileName&wrapper=artifacts" | grep -o "[^<>]*cp${MAJOR}${MINOR}[^<>]*.whl"); \ PACKAGE=$(wget -qO- ${BASE}"api/xml?xpath=//fileName&wrapper=artifacts" | grep -o "[^<>]*cp${MAJOR}${MINOR}[^<>]*.whl"); \
wget ${BASE}"artifact/tensorflow_pkg/"${PACKAGE}; \ wget ${BASE}"artifact/tensorflow_pkg/"${PACKAGE}; \
${PIP} install ${PACKAGE} ${PIP} install ${PACKAGE}
@ -126,10 +120,9 @@ RUN if [ ${TF_PACKAGE} = tensorflow-gpu ]; then \
COPY bashrc /etc/bash.bashrc COPY bashrc /etc/bash.bashrc
RUN chmod a+rwx /etc/bash.bashrc RUN chmod a+rwx /etc/bash.bashrc
RUN ${PIP} install jupyter matplotlib RUN python3 -m pip install jupyter matplotlib
# Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422 # Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422
RUN if [[ "${USE_PYTHON_3_NOT_2}" == "1" ]]; then ${PIP} install ipykernel==5.1.1 nbformat==4.4.0; fi RUN python3 -m pip install jupyter_http_over_ws ipykernel==5.1.1 nbformat==4.4.0
RUN ${PIP} install jupyter_http_over_ws
RUN jupyter serverextension enable --py jupyter_http_over_ws RUN jupyter serverextension enable --py jupyter_http_over_ws
RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/ RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/
@ -149,6 +142,6 @@ RUN apt-get autoremove -y && apt-get remove -y wget
WORKDIR /tf WORKDIR /tf
EXPOSE 8888 EXPOSE 8888
RUN ${PYTHON} -m ipykernel.kernelspec RUN python3 -m ipykernel.kernelspec
CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"] CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"]

18
tensorflow/tools/dockerfiles/dockerfiles/ppc64le/gpu-ppc64le.Dockerfile
View File

@ -74,25 +74,19 @@ RUN ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/lib
&& echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \ && echo "/usr/local/cuda/lib64/stubs" > /etc/ld.so.conf.d/z-cuda-stubs.conf \
&& ldconfig && ldconfig
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python
# Options: # Options:
# tensorflow # tensorflow
@ -117,8 +111,8 @@ RUN if [ ${TF_PACKAGE} = tensorflow-gpu ]; then \
elif [ ${TF_PACKAGE} = tf-nightly ]; then \ elif [ ${TF_PACKAGE} = tf-nightly ]; then \
BASE=https://powerci.osuosl.org/job/TensorFlow_PPC64LE_CPU_Nightly_Artifact/lastSuccessfulBuild/; \ BASE=https://powerci.osuosl.org/job/TensorFlow_PPC64LE_CPU_Nightly_Artifact/lastSuccessfulBuild/; \
fi; \ fi; \
MAJOR=`${PYTHON} -c 'import sys; print(sys.version_info[0])'`; \ MAJOR=`python3 -c 'import sys; print(sys.version_info[0])'`; \
MINOR=`${PYTHON} -c 'import sys; print(sys.version_info[1])'`; \ MINOR=`python3 -c 'import sys; print(sys.version_info[1])'`; \
PACKAGE=$(wget -qO- ${BASE}"api/xml?xpath=//fileName&wrapper=artifacts" | grep -o "[^<>]*cp${MAJOR}${MINOR}[^<>]*.whl"); \ PACKAGE=$(wget -qO- ${BASE}"api/xml?xpath=//fileName&wrapper=artifacts" | grep -o "[^<>]*cp${MAJOR}${MINOR}[^<>]*.whl"); \
wget ${BASE}"artifact/tensorflow_pkg/"${PACKAGE}; \ wget ${BASE}"artifact/tensorflow_pkg/"${PACKAGE}; \
${PIP} install ${PACKAGE} ${PIP} install ${PACKAGE}

7
tensorflow/tools/dockerfiles/partials/jupyter.partial.Dockerfile
View File

@ -1,7 +1,6 @@
RUN ${PIP} install jupyter matplotlib RUN python3 -m pip install jupyter matplotlib
# Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422 # Pin ipykernel and nbformat; see https://github.com/ipython/ipykernel/issues/422
RUN if [[ "${USE_PYTHON_3_NOT_2}" == "1" ]]; then ${PIP} install ipykernel==5.1.1 nbformat==4.4.0; fi RUN python3 -m pip install jupyter_http_over_ws ipykernel==5.1.1 nbformat==4.4.0
RUN ${PIP} install jupyter_http_over_ws
RUN jupyter serverextension enable --py jupyter_http_over_ws RUN jupyter serverextension enable --py jupyter_http_over_ws
RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/ RUN mkdir -p /tf/tensorflow-tutorials && chmod -R a+rwx /tf/
@ -21,6 +20,6 @@ RUN apt-get autoremove -y && apt-get remove -y wget
WORKDIR /tf WORKDIR /tf
EXPOSE 8888 EXPOSE 8888
RUN ${PYTHON} -m ipykernel.kernelspec RUN python3 -m ipykernel.kernelspec
CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"] CMD ["bash", "-c", "source /etc/bash.bashrc && jupyter notebook --notebook-dir=/tf --ip 0.0.0.0 --no-browser --allow-root"]

4
tensorflow/tools/dockerfiles/partials/tensorflow-ppc64le.partial.Dockerfile
View File

@ -21,8 +21,8 @@ RUN if [ ${TF_PACKAGE} = tensorflow-gpu ]; then \
elif [ ${TF_PACKAGE} = tf-nightly ]; then \ elif [ ${TF_PACKAGE} = tf-nightly ]; then \
BASE=https://powerci.osuosl.org/job/TensorFlow_PPC64LE_CPU_Nightly_Artifact/lastSuccessfulBuild/; \ BASE=https://powerci.osuosl.org/job/TensorFlow_PPC64LE_CPU_Nightly_Artifact/lastSuccessfulBuild/; \
fi; \ fi; \
MAJOR=`${PYTHON} -c 'import sys; print(sys.version_info[0])'`; \ MAJOR=`python3 -c 'import sys; print(sys.version_info[0])'`; \
MINOR=`${PYTHON} -c 'import sys; print(sys.version_info[1])'`; \ MINOR=`python3 -c 'import sys; print(sys.version_info[1])'`; \
PACKAGE=$(wget -qO- ${BASE}"api/xml?xpath=//fileName&wrapper=artifacts" | grep -o "[^<>]*cp${MAJOR}${MINOR}[^<>]*.whl"); \ PACKAGE=$(wget -qO- ${BASE}"api/xml?xpath=//fileName&wrapper=artifacts" | grep -o "[^<>]*cp${MAJOR}${MINOR}[^<>]*.whl"); \
wget ${BASE}"artifact/tensorflow_pkg/"${PACKAGE}; \ wget ${BASE}"artifact/tensorflow_pkg/"${PACKAGE}; \
${PIP} install ${PACKAGE} ${PIP} install ${PACKAGE}

3
tensorflow/tools/dockerfiles/partials/ubuntu/bazel.partial.Dockerfile
View File

@ -4,7 +4,7 @@ RUN apt-get update && apt-get install -y \
git \ git \
wget \ wget \
openjdk-8-jdk \ openjdk-8-jdk \
${PYTHON}-dev \ python3-dev \
virtualenv \ virtualenv \
swig swig
@ -20,7 +20,6 @@ RUN ${PIP} --no-cache-dir install \
pandas \ pandas \
future \ future \
portpicker \ portpicker \
&& test "${USE_PYTHON_3_NOT_2}" -eq 1 && true || ${PIP} --no-cache-dir install \
enum34 enum34
# Install bazel # Install bazel

3
tensorflow/tools/dockerfiles/partials/ubuntu/bazelbuild.partial.Dockerfile
View File

@ -3,7 +3,7 @@ RUN apt-get update && apt-get install -y \
curl \ curl \
git \ git \
openjdk-8-jdk \ openjdk-8-jdk \
${PYTHON}-dev \ python3-dev \
virtualenv \ virtualenv \
swig swig
@ -18,7 +18,6 @@ RUN ${PIP} --no-cache-dir install \
sklearn \ sklearn \
pandas \ pandas \
portpicker \ portpicker \
&& test "${USE_PYTHON_3_NOT_2}" -eq 1 && true || ${PIP} --no-cache-dir install \
enum34 enum34
# Build and install bazel # Build and install bazel

14
tensorflow/tools/dockerfiles/partials/ubuntu/python.partial.Dockerfile
View File

@ -1,19 +1,13 @@
ARG USE_PYTHON_3_NOT_2
# TODO(angerson) Completely remove Python 2 support
ARG _PY_SUFFIX=${USE_PYTHON_3_NOT_2:+3}
ARG PYTHON=python${_PY_SUFFIX}
ARG PIP=pip${_PY_SUFFIX}
# See http://bugs.python.org/issue19846 # See http://bugs.python.org/issue19846
ENV LANG C.UTF-8 ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y \ RUN apt-get update && apt-get install -y \
${PYTHON} \ python3
${PYTHON}-pip python3-pip
RUN ${PIP} --no-cache-dir install --upgrade \ RUN python3 -m pip --no-cache-dir install --upgrade \
pip \ pip \
setuptools setuptools
# Some TF tools expect a "python" binary # Some TF tools expect a "python" binary
RUN ln -s $(which ${PYTHON}) /usr/local/bin/python RUN ln -s $(which python3) /usr/local/bin/python

31
tensorflow/tools/dockerfiles/spec.yml
View File

@ -25,28 +25,19 @@ header: |
# of functionality ("slices") by listing all of the "slice sets" to use when # of functionality ("slices") by listing all of the "slice sets" to use when
# building. # building.
# #
# For example, a release that uses {nightly}{py} would create 4 Dockerfiles
# (which could become images or concrete Dockerfiles), because the "nightly"
# and "py" slice sets both have two entries:
#
# - nightly (no -py2 because the Python 2 slice set has add_to_name: ""
# - nightly-py3
# - nightly-gpu (similar)
# - nightly-gpu-py3
#
# Releases are all treated differently by TensorFlow's CI systems. # Releases are all treated differently by TensorFlow's CI systems.
releases: releases:
# Built Nightly and pushed to tensorflow/tensorflow # Built Nightly and pushed to tensorflow/tensorflow
nightly: nightly:
tag_specs: tag_specs:
- "{nightly}{py}{jupyter}" - "{nightly}{jupyter}"
- "{_TAG_PREFIX}{ubuntu-devel}{py-devel}" - "{_TAG_PREFIX}{ubuntu-devel}"
# Built per-release and pushed to tensorflow/tensorflow # Built per-release and pushed to tensorflow/tensorflow
# --arg _TAG_PREFIX=<val> should be set to "1.11" (for example) or "latest". # --arg _TAG_PREFIX=<val> should be set to "1.11" (for example) or "latest".
versioned: versioned:
tag_specs: tag_specs:
- "{_TAG_PREFIX}{ubuntu}{py}{jupyter}" - "{_TAG_PREFIX}{ubuntu}{jupyter}"
# Dockerfiles stored in the TF repo; not pushed anywhere # Dockerfiles stored in the TF repo; not pushed anywhere
dockerfiles: dockerfiles:
@ -62,22 +53,6 @@ releases:
slice_sets: slice_sets:
py:
- add_to_name: ""
args:
- USE_PYTHON_3_NOT_2=1
- add_to_name: "-py3"
args:
- USE_PYTHON_3_NOT_2=1
py-devel:
- add_to_name: ""
args:
- USE_PYTHON_3_NOT_2=1
- add_to_name: "-py3"
args:
- USE_PYTHON_3_NOT_2=1
jupyter: jupyter:
- add_to_name: "" - add_to_name: ""
- add_to_name: "-jupyter" - add_to_name: "-jupyter"