Skip to content

perf(docker): optimize Dockerfiles with BuildKit cache for faster rebuilds #109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fujikosu merged 7 commits into from
Dec 16, 2025
Merged

perf(docker): optimize Dockerfiles with BuildKit cache for faster rebuilds #109

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
96ce38a
perf(docker): optimize Dockerfiles with BuildKit cache for faster reb…
fujikosu Dec 14, 2025
891ea72
perf(docker): enhance Dockerfiles with BuildKit cache for improved pe…
fujikosu Dec 14, 2025
e2006d0
perf(docker): Use `--cache-dir` flag instead of PIP_CACHE_DIR env var…
fujikosu Dec 14, 2025
b92366e
Update src/sample_cpu_project/.devcontainer/Dockerfile
fujikosu Dec 14, 2025
3a96f28
Update notebooks/.devcontainer/Dockerfile
fujikosu Dec 14, 2025
219571e
perf(docker): update Dockerfiles to use /root/.cache/pip for pip caching
fujikosu Dec 15, 2025
c1bac6b
fix(docker): add --yes flag to az extension installation for non-inte…
fujikosu Dec 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 17 additions & 10 deletions notebooks/.devcontainer/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,28 @@ ARG USERNAME=devuser
ARG USER_UID=1000
ARG USER_GID=$USER_UID
RUN groupadd --gid $USER_GID $USERNAME \
&& useradd --uid $USER_UID --gid $USER_GID -m $USERNAME \
# Add sudo support
&& apt-get update \
&& useradd --uid $USER_UID --gid $USER_GID -m $USERNAME

# Add sudo support (using BuildKit cache mounts to speed up rebuilds)
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
apt-get update \
&& apt-get install -y sudo \
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
&& chmod 0440 /etc/sudoers.d/$USERNAME \
&& rm -rf /var/lib/apt/lists/*
Comment thread
fujikosu marked this conversation as resolved.
&& chmod 0440 /etc/sudoers.d/$USERNAME
USER $USERNAME

# make all python tools installed by pip accesible
# Make pip-installed tools accessible
ENV PATH=$PATH:/home/$USERNAME/.local/bin
RUN pip install --no-cache-dir pip --upgrade

# Install development tools (linters, formatters, test runners, etc.)
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --cache-dir=/root/.cache/pip pip --upgrade
COPY requirements-dev.txt .
RUN pip install --no-cache-dir -r requirements-dev.txt
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --cache-dir=/root/.cache/pip -r requirements-dev.txt

# install notebooks related depencencies
# Install notebooks related dependencies
COPY notebooks/.devcontainer/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --cache-dir=/root/.cache/pip -r requirements.txt
1 change: 1 addition & 0 deletions notebooks/.devcontainer/devcontainer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{
"name": "DSToolkit Notebooks Dev Container",
"build": {
// use root directory as build context so that requirements-dev.txt is accessible during build
"context": "../../",
Expand Down
35 changes: 21 additions & 14 deletions src/sample_cpu_project/.devcontainer/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,36 @@ ARG USERNAME=devuser
ARG USER_UID=1000
ARG USER_GID=$USER_UID
RUN groupadd --gid $USER_GID $USERNAME \
&& useradd --uid $USER_UID --gid $USER_GID -m $USERNAME \
# Add sudo support
&& apt-get update \
&& useradd --uid $USER_UID --gid $USER_GID -m $USERNAME

# Add sudo support (using BuildKit cache mounts to speed up rebuilds)
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
apt-get update \
&& apt-get install -y sudo \
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
&& chmod 0440 /etc/sudoers.d/$USERNAME \
&& rm -rf /var/lib/apt/lists/*
Comment thread
fujikosu marked this conversation as resolved.
&& chmod 0440 /etc/sudoers.d/$USERNAME
USER $USERNAME

# make all python tools installed by pip accesible
ENV PATH=$PATH:/home/devuser/.local/bin
# Make pip-installed tools accessible
ENV PATH=$PATH:/home/$USERNAME/.local/bin

RUN pip install --no-cache-dir pip --upgrade
# Install Python packages (using BuildKit cache mounts for faster rebuilds)
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --cache-dir=/root/.cache/pip pip --upgrade

COPY src/sample_cpu_project/.devcontainer/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --cache-dir=/root/.cache/pip -r requirements.txt

# install common module related pacakages
COPY src/common/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Install common module related packages
COPY src/common/requirements.txt common-requirements.txt
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --cache-dir=/root/.cache/pip -r common-requirements.txt

# install python tools
# Install development tools (linters, formatters, test runners, etc.)
COPY requirements-dev.txt .
RUN pip install --no-cache-dir -r requirements-dev.txt
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --cache-dir=/root/.cache/pip -r requirements-dev.txt


1 change: 1 addition & 0 deletions src/sample_cpu_project/.devcontainer/devcontainer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{
"name": "Sample CPU Project Dev Container",
"build": {
// use root directory as build context so that requirements-dev.txt is accessible during build
"context": "../../../",
Expand Down
36 changes: 21 additions & 15 deletions src/sample_pytorch_gpu_project/.devcontainer/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,30 +6,36 @@ ARG USERNAME=devuser
ARG USER_UID=1000
ARG USER_GID=$USER_UID
RUN groupadd --gid $USER_GID $USERNAME \
&& useradd --uid $USER_UID --gid $USER_GID -m $USERNAME \
# Add sudo support
&& apt-get update \
&& useradd --uid $USER_UID --gid $USER_GID -m $USERNAME

# Add sudo support (using BuildKit cache mounts to speed up rebuilds)
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
apt-get update \
&& apt-get install -y sudo \
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
&& chmod 0440 /etc/sudoers.d/$USERNAME \
&& rm -rf /var/lib/apt/lists/*
Comment thread
fujikosu marked this conversation as resolved.
&& chmod 0440 /etc/sudoers.d/$USERNAME
USER $USERNAME


# make all python tools installed by pip accesible
# Make pip-installed tools accessible
ENV PATH=$PATH:/home/$USERNAME/.local/bin

RUN pip install pip --upgrade
# Install Python packages (using BuildKit cache mounts for faster rebuilds)
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --cache-dir=/root/.cache/pip pip --upgrade

COPY src/sample_pytorch_gpu_project/.devcontainer/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --cache-dir=/root/.cache/pip -r requirements.txt

# install common module related pacakages
COPY src/common/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Install common module related packages
COPY src/common/requirements.txt common-requirements.txt
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --cache-dir=/root/.cache/pip -r common-requirements.txt

# install python tools
# Install development tools (linters, formatters, test runners, etc.)
COPY requirements-dev.txt .
RUN pip install --no-cache-dir -r requirements-dev.txt
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --cache-dir=/root/.cache/pip -r requirements-dev.txt

RUN az extension add --name ml
RUN az extension add --name ml --yes
1 change: 1 addition & 0 deletions src/sample_pytorch_gpu_project/.devcontainer/devcontainer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{
"name": "Sample PyTorch GPU Project Dev Container",
"build": {
// use root directory as build context so that requirements-dev.txt is accessible during build
"context": "../../../",
Expand Down