Skip to content

Build ray 2.0.0 dev #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Build ray 2.0.0 dev #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
71dff6d
build ray 2.2.0 dev
Leemoonsoo Mar 7, 2021
1e23809
copy all requirements.txt
Leemoonsoo Mar 7, 2021
e848252
update path requirements_tun.txt
Leemoonsoo Mar 7, 2021
f54fc2e
run test on branch ray-2.0.0
Leemoonsoo Mar 7, 2021
0fb7bf2
try skip install tensorflow-gpu on CI when python version is 3.6.x to…
Leemoonsoo Mar 7, 2021
be435fa
define SED_INPLACE
Leemoonsoo Mar 7, 2021
081fbe9
define SED_INPLACE in the beginning of the file
Leemoonsoo Mar 7, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build-docker-images.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ on:
tags-ignore:
- '*.*'
branches:
- master
- main
pull_request:

jobs:
Expand Down
26 changes: 20 additions & 6 deletions .staroid/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,19 @@
# https://skaffold.dev/docs/pipeline-stages/builders/custom/
#

# sed
SED_INPLACE="sed -i"
uname | grep Darwin > /dev/null
if [ $? -eq 0 ]; then
SED_INPLACE="sed -i .bak"
fi

set -x
set -e
pwd

RAY_REPO=https://github.com/ray-project/ray.git
RAY_CHECKOUT=260b07cf0cf2c10c091711cc3d598663133c2dc5
RAY_CHECKOUT=5fc761c562006432ed6fd43066ba3d3c7b9c6ac5
PYTHON_VERSION=$1
SHORT_VER=`echo $PYTHON_VERSION | sed "s/\([0-9]*\)[.]\([0-9]*\)[.][0-9]*/\1\2/g"`

Expand Down Expand Up @@ -61,12 +68,12 @@ if [ "$BUILD_WHEEL" == "true" ]; then

WHEEL=`ls .whl/*-cp$SHORT_VER-*`
else
if [ "$SHORT_VER" == "36" ]; then
WHEEL="https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp36-cp36m-manylinux1_x86_64.whl"
if [ "$SHORT_VER" == "36" ]; then
WHEEL="https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-2.0.0.dev0-cp36-cp36m-manylinux2014_x86_64.whl"
elif [ "$SHORT_VER" == "37" ]; then
WHEEL="https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp37-cp37m-manylinux1_x86_64.whl"
WHEEL="https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-2.0.0.dev0-cp37-cp37m-manylinux2014_x86_64.whl"
elif [ "$SHORT_VER" == "38" ]; then
WHEEL="https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp38-cp38-manylinux1_x86_64.whl"
WHEEL="https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-2.0.0.dev0-cp38-cp38-manylinux2014_x86_64.whl"
fi
fi

Expand All @@ -75,8 +82,15 @@ fi
../.staroid/ray_patch.sh patch . $WHEEL

# apply additional docker file commands
if [ "$SHORT_VER" == "36" ] && [ "$GITHUB_ACTIONS" == "true" ]; then
# when github action runs python36 build, it stops with no space left error during installing conda install tensorflow-gpu
# so in this case, we skip install tensorflow-gpu package install
$SED_INPLACE "/tensorflow/d" ../.staroid/Dockerfile_staroid
fi

cat ../.staroid/Dockerfile_staroid >> docker/ray-ml/Dockerfile


# print patched files
git diff

Expand All @@ -86,7 +100,7 @@ cat docker/ray/Dockerfile
cat docker/ray-ml/Dockerfile

# cp requirements to ray-ml dir
cp python/requirements* docker/ray-ml
cp python/requirements.txt python/requirements_* python/requirements/* docker/ray-ml

# build docker image
./build-docker.sh --no-cache-build --gpu --python-version $PYTHON_VERSION
Expand Down
4 changes: 2 additions & 2 deletions .staroid/ray_patch.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ if [ "$OP" == "patch" ]; then
$SED_INPLACE "/tensorflow/d" ${RAY_HOME}/python/requirements.txt
$SED_INPLACE "/tensorflow/d" ${RAY_HOME}/python/requirements_ml_docker.txt
$SED_INPLACE "/tensorflow/d" ${RAY_HOME}/python/requirements_rllib.txt
$SED_INPLACE "/tensorflow/d" ${RAY_HOME}/python/requirements_tune.txt
$SED_INPLACE "/tensorflow/d" ${RAY_HOME}/python/requirements/requirements_tune.txt

elif [ "$OP" == "reset" ]; then
git checkout ${RAY_HOME}/docker/ray/Dockerfile
Expand All @@ -73,7 +73,7 @@ elif [ "$OP" == "reset" ]; then
git checkout ${RAY_HOME}/python/requirements.txt
git checkout ${RAY_HOME}/python/requirements_ml_docker.txt
git checkout ${RAY_HOME}/python/requirements_rllib.txt
git checkout ${RAY_HOME}/python/requirements_tune.txt
git checkout ${RAY_HOME}/python/requirements/requirements_tune.txt
else
echo "Invalid operation $OP"
exit 1
Expand Down