Skip to content

speed up image builds #58

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
e-carlin opened this issue Feb 7, 2025 · 5 comments
Open

speed up image builds #58

e-carlin opened this issue Feb 7, 2025 · 5 comments

Comments

@e-carlin
Copy link
Collaborator

e-carlin commented Feb 7, 2025

Compiling epics take a while. That code is unlikely to change so it doesn't make sense to wait for it at each build.

Some ideas:

  • Create an epics base image and then layer slicops on top
  • Create an epics rpm and install it.
@robnagler
Copy link
Collaborator

I noticed a couple of things:

  • Remove motif or libXt. This will make the image smaller, maybe by a lot, because this brings in all of X11.
  • The base epics build can be parallelized. It may be the ASYN build cannot so you can build that later, I think.
  • We may not need to build ASYN separately. It was needed for the medm modules, which are no longer needed. We might want to use the one with SYNAPPS, this definitely can be parallel.

@robnagler
Copy link
Collaborator

robnagler commented Feb 12, 2025

Could maybe remove the testApp/Ca parts as well. There are other tests such as ntndarrayTest. The linking is a slow part.

@robnagler
Copy link
Collaborator

We already have an epics rpm. We could add SYNAPPS to it.

@robnagler
Copy link
Collaborator

robnagler commented Feb 13, 2025

Tried to use rscode-epics and rscode-epics-asyn, but got this strange error building synapps:

#7 206.3 /usr/bin/g++  -D_GNU_SOURCE -D_DEFAULT_SOURCE        -DUSE_TYPED_RSET  -D_X86_64_ -DUNIX  -Dlinux      -O3 -g   -Wall      -mtune=generic     -m64  -fPIC -I. -I../O.Common -I.\
 -I. -I.. -I../../../include/compiler/gcc -I../../../include/os/Linux -I../../../include        -I/home/vagrant/.local/epics/extensions/synApps/support/autosave-R5-11/include/os/Linux \
-I/home/vagrant/.local/epics/extensions/synApps/support/autosave-R5-11/include    -I/home/vagrant/.local/epics/include/compiler/gcc -I/home/vagrant/.local/epics/include/os/Linux -I/hom\
e/vagrant/.local/epics/include        -MM -MF testBusyAsyn.d  ../testBusyAsyn.cpp
#7 206.3 /usr/bin/gcc  -D_GNU_SOURCE -D_DEFAULT_SOURCE        -DUSE_TYPED_RSET  -D_X86_64_ -DUNIX  -Dlinux      -O3 -g   -Wall -Werror-implicit-function-declaration      -mtune=generic\
     -m64  -fPIC -I. -I../O.Common -I. -I. -I.. -I../../../include/compiler/gcc -I../../../include/os/Linux -I../../../include        -I/home/vagrant/.local/epics/extensions/synApps/su\
pport/autosave-R5-11/include/os/Linux -I/home/vagrant/.local/epics/extensions/synApps/support/autosave-R5-11/include    -I/home/vagrant/.local/epics/include/compiler/gcc -I/home/vagran\
t/.local/epics/include/os/Linux -I/home/vagrant/.local/epics/include        -MM -MF devBusyAsyn.d  ../devBusyAsyn.c
#7 206.4 perl -CSD /home/vagrant/.local/epics/bin/linux-x86_64/makeMakefile.pl O.linux-x86_64 ../../..
#7 206.4 mkdir -p O.Common
#7 206.4 ../devBusyAsyn.c:45:10: fatal error: asynEpicsUtils.h: No such file or directory
#7 206.4    45 | #include "asynEpicsUtils.h"
#7 206.4       |          ^~~~~~~~~~~~~~~~~~
#7 206.4 compilation terminated.
#7 206.4 make -C O.linux-x86_64 -f ../Makefile TOP=../../.. \
#7 206.4     T_A=linux-x86_64 install
#7 206.4 /usr/bin/gcc  -D_GNU_SOURCE -D_DEFAULT_SOURCE        -DUSE_TYPED_RSET  -D_X86_64_ -DUNIX  -Dlinux      -O3 -g   -Wall -Werror-implicit-function-declaration      -mtune=generic\
     -m64  -fPIC -I. -I../O.Common -I. -I. -I.. -I../../../include/compiler/gcc -I../../../include/os/Linux -I../../../include        -I/home/vagrant/.local/epics/extensions/synApps/su\
pport/autosave-R5-11/include/os/Linux -I/home/vagrant/.local/epics/extensions/synApps/support/autosave-R5-11/include    -I/home/vagrant/.local/epics/include/compiler/gcc -I/home/vagran\
t/.local/epics/include/os/Linux -I/home/vagrant/.local/epics/include        -MM -MF devBusySoftRaw.d  ../devBusySoftRaw.c
#7 206.4 make[4]: Entering directory '/home/vagrant/.local/epics/extensions/synApps/support/sequencer-mirror-R2-2-9/src/snc/O.linux-x86_64'
#7 206.4 dbdExpand.pl: Can't find file 'asynRecord.dbd'
#7 206.4   while reading '../testBusyAsynInclude.dbd' to create 'testBusyAsyn.dbd'
#7 206.4 dbdExpand.pl: Exiting due to errors
#7 206.4 make[4]: *** [/home/vagrant/.local/epics/configure/RULES.Db:388: ../O.Common/testBusyAsyn.dbd] Error 2
diff --git a/container-conf/build.sh b/container-conf/build.sh
index 4efc572..f71a88f 100644
--- a/container-conf/build.sh
+++ b/container-conf/build.sh
@@ -8,27 +8,23 @@ build_vars() {
 }

 build_as_root() {
-    build_yum install \
-        libXt-devel \
-        libtirpc-devel \
-        motif-devel \
+    install_repo_eval code epics epics-asyn
+    # re2c is for synapps
+    install_yum_install \
         nodejs \
         perl-ExtUtils-Command \
         perl-FindBin \
-        rpcgen
+        re2c
 }

 build_as_run_user() {
-    install_source_bashrc
     cat > ~/.post_bivio_bashrc <<'EOF'
 export EPICS_BASE=$HOME/.local/epics
 # $EPICS_BASE/startup/EpicsHostArch outputs linux-x86_64; no need to be dynamic here
 export EPICS_HOST_ARCH=linux-x86_64
 bivio_path_insert "$EPICS_BASE/bin/$EPICS_HOST_ARCH"
-f=$EPICS_BASE/extensions/synApps/support/areaDetector-R3-12-1
-export EPICS_DISPLAY_PATH=.:$f/ADSimDetector/simDetectorApp/op/adl:$f/ADCore/ADApp/op/adl:$f/ADUVC/uvcApp/op/adl:$EPICS_BASE/modules/asyn/asyn-R4-45/opi/medm
 EOF
-    install_not_strict_cmd source ~/.post_bivio_bashrc
+    install_source_bashrc
     source epics-install.sh
     cd ~/src/
     _slicops_pip_install radiasoft/pykern "$PYKERN_BRANCH"
diff --git a/container-conf/epics-install.sh b/container-conf/epics-install.sh
index decc1f8..b266006 100644
--- a/container-conf/epics-install.sh
+++ b/container-conf/epics-install.sh
@@ -6,36 +6,8 @@ set -eou pipefail
 shopt -s nullglob

 _asyn_version=R4-45
-_epics_version=7.0.8.1
 _synapps_version=R6-3

-_curl_untar() {
-    declare url=$1
-    declare base=$2
-    declare tgt=$3
-    curl -L -s -S "$url" | tar xzf -
-    mv "$base" "$tgt"
-    cd "$tgt"
-}
-
-_build_base_and_asyn() {
-    sudo dnf -y install re2c
-    declare d=$(dirname "$EPICS_BASE")
-    mkdir -p "$d"
-    cd "$d"
-    b=base-"$_epics_version"
-    _curl_untar https://epics-controls.org/download/base/"$b".tar.gz "$b" "$EPICS_BASE"
-    cd "$EPICS_BASE"
-    cd modules
-    _curl_untar https://github.com/epics-modules/asyn/archive/refs/tags/"$_asyn_version".tar.gz "asyn-$_asyn_version" asyn
-    perl -pi -e 's/^# (?=TIRPC)//' configure/CONFIG_SITE
-    cd ..
-    echo 'SUBMODULES += asyn' > Makefile.local
-    cd ..
-    # parallel make does not work
-    make
-}
-
 _build_synapps() {
     declare d=synApps
     # Must be absolute or fails silently
@@ -62,7 +34,6 @@ EOF

 _main() {
     bivio_path_remove "$EPICS_BASE"/bin
-    _build_base_and_asyn
     cd "$EPICS_BASE"
     mkdir -p extensions
     cd extensions

robnagler added a commit that referenced this issue Feb 19, 2025
- Partial #58 removed unnecessary dnf installs; parallelize epics
- Read st_cmd and send to stdin so don't have to chdir
- added slicops-demo and print instructions
- npm audit fix --force updated build-angular to 19.1.7 from 16.2.16
- added device_db_test 
- allow dicts to raise errors
- set build_image_name to radiasoft/ so can push
@robnagler
Copy link
Collaborator

vue js helps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants