Add SMUS patches to Code Editor

[minshinz]

Description

Similar to how we apply smus patches in entrypoint-sagemaker-ui-jupyter-server, we should do the same for entrypoint-sagemaker-ui-code-editor. All these patches need to be applied.

This will fix the issues in CE

1. EMR computes didn't return the right session table.

Description

The spark UI/ Driver Logs links are shown as below, currently the functionality is broken is VS Code. One of reason is that patch files currently are not applied to codeditor.

Type of Change

• [ x ] Image update - Bug fix
• Image update - New feature
• Image update - Breaking change
• SMD image build tool update
• Documentation update

Release Information

Does this change need to be included in patch version releases? By default, any pull requests will only be added to the next SMD image minor version release once they are merged in template folder. Only critical bug fix or security update should be applied to new patch versions of existed image minor versions.

• [ x ] Yes (Critical bug fix or security update)
• No (New feature or non-critical change)
• N/A (Not an image update)

If yes, please explain why:
[Explain the criticality of this change and why it should be included in patch releases]

How Has This Been Tested?

Manually Tested in Code Editor env and ensure the patches apply

sagemaker-user@default:/etc/patches/smus-code-editor$ sudo vim 0001-sparkmagic.patch
sagemaker-user@default:/etc/patches/smus-code-editor$ sudo vim 0002-sparkmagic-override-info-table.patch
sagemaker-user@default:/etc/patches/smus-code-editor$ cd ..
sagemaker-user@default:/etc/patches$ ls
apply_patches.sh  smus  smus-code-editor
sagemaker-user@default:/etc/patches$ sudo rm apply_patches.sh
sagemaker-user@default:/etc/patches$ sudo vim apply_patches.sh
sagemaker-user@default:/etc/patches$ sudo chmod +x /etc/patches/apply_patches.sh
sagemaker-user@default:/etc/patches$ /etc/patches/apply_patches.sh smus-code-editor

Confrimed files are successfully Patched.

Tested in 2.8, both patch3/4 are skipped as expected.

/etc/patches/apply_patches.sh smus
+ '[' 1 -ne 1 ']'
+ case "$1" in
+ PATCH_DIR=/etc/patches/smus
+ '[' '!' -d /etc/patches/smus ']'
+ for PATCHFILE in "$PATCH_DIR"/*.patch
+ '[' -f /etc/patches/smus/0003-fix-ipython-display.patch ']'
+ should_skip_patch /etc/patches/smus/0003-fix-ipython-display.patch
+ local patch_file=/etc/patches/smus/0003-fix-ipython-display.patch
++ basename /etc/patches/smus/0003-fix-ipython-display.patch
+ local patch_basename=0003-fix-ipython-display.patch
+ [[ 0003-fix-ipython-display.patch == *\f\i\x\-\i\p\y\t\h\o\n\-\d\i\s\p\l\a\y* ]]
++ get_package_version hdijupyterutils
++ local package_name=hdijupyterutils
+++ pip show hdijupyterutils
+++ cut '-d ' -f2
+++ grep Version:
++ local pkg_version=0.23.0
++ '[' -z 0.23.0 ']'
++ echo 0.23.0
+ local hdijupyterutils_version=0.23.0
+ '[' -n 0.23.0 ']'
+ version_gte 0.23.0 0.23.0
+ local version1=0.23.0
+ local version2=0.23.0
+ '[' -z 0.23.0 ']'
+ '[' -z 0.23.0 ']'
+ '[' 0.23.0 = 0.23.0 ']'
+ return 0
+ echo 'Skipping 0003-fix-ipython-display.patch: hdijupyterutils version 0.23.0 >= 0.23'
Skipping 0003-fix-ipython-display.patch: hdijupyterutils version 0.23.0 >= 0.23
+ return 0
+ continue
+ for PATCHFILE in "$PATCH_DIR"/*.patch
+ '[' -f /etc/patches/smus/0004-fix-boto3-endpoints.patch ']'
+ should_skip_patch /etc/patches/smus/0004-fix-boto3-endpoints.patch
+ local patch_file=/etc/patches/smus/0004-fix-boto3-endpoints.patch
++ basename /etc/patches/smus/0004-fix-boto3-endpoints.patch
+ local patch_basename=0004-fix-boto3-endpoints.patch
+ [[ 0004-fix-boto3-endpoints.patch == *\f\i\x\-\i\p\y\t\h\o\n\-\d\i\s\p\l\a\y* ]]
+ [[ 0004-fix-boto3-endpoints.patch == *\f\i\x\-\b\o\t\o\3\-\e\n\d\p\o\i\n\t\s* ]]
++ get_package_version botocore
++ local package_name=botocore
+++ pip show botocore
+++ cut '-d ' -f2
+++ grep Version:
++ local pkg_version=1.38.46
++ '[' -z 1.38.46 ']'
++ echo 1.38.46
+ local botocore_version=1.38.46
+ '[' -n 1.38.46 ']'
+ version_gte 1.38.46 1.37.17
+ local version1=1.38.46
+ local version2=1.37.17
+ '[' -z 1.38.46 ']'
+ '[' -z 1.37.17 ']'
+ '[' 1.38.46 = 1.37.17 ']'
++ printf '%s\n%s\n' 1.38.46 1.37.17
++ sort -V
+ local 'sorted=1.37.17
1.38.46'
++ echo '1.37.17
1.38.46'
++ head -n1
+ local first_line=1.37.17
+ '[' 1.37.17 = 1.37.17 ']'
+ echo 'Skipping 0004-fix-boto3-endpoints.patch: botocore version 1.38.46 >= 1.37.17'
Skipping 0004-fix-boto3-endpoints.patch: botocore version 1.38.46 >= 1.37.17
+ return 0
+ continue

Tested in 3.1, patch 3 is skipped and 4 is applied as expected.

+ for PATCHFILE in "$PATCH_DIR"/*.patch
+ '[' -f /etc/patches/smus/0003-fix-ipython-display.patch ']'
+ should_skip_patch /etc/patches/smus/0003-fix-ipython-display.patch
+ local patch_file=/etc/patches/smus/0003-fix-ipython-display.patch
++ basename /etc/patches/smus/0003-fix-ipython-display.patch
+ local patch_basename=0003-fix-ipython-display.patch
+ [[ 0003-fix-ipython-display.patch == *\f\i\x\-\i\p\y\t\h\o\n\-\d\i\s\p\l\a\y* ]]
++ get_package_version hdijupyterutils
++ local package_name=hdijupyterutils
+++ pip show hdijupyterutils
+++ cut '-d ' -f2
+++ grep Version:
++ local pkg_version=0.22.0
++ '[' -z 0.22.0 ']'
++ echo 0.22.0
+ local hdijupyterutils_version=0.22.0
+ '[' -n 0.22.0 ']'
+ version_gte 0.22.0 0.23.0
+ local version1=0.22.0
+ local version2=0.23.0
+ '[' -z 0.22.0 ']'
+ '[' -z 0.23.0 ']'
+ '[' 0.22.0 = 0.23.0 ']'
++ printf '%s\n%s\n' 0.22.0 0.23.0
++ sort -V
+ local 'sorted=0.22.0
0.23.0'
++ head -n1
++ echo '0.22.0
0.23.0'
+ local first_line=0.22.0
+ '[' 0.22.0 = 0.23.0 ']'
+ [[ 0003-fix-ipython-display.patch == *\f\i\x\-\b\o\t\o\3\-\e\n\d\p\o\i\n\t\s* ]]
+ return 1
+ echo 'Applying /etc/patches/smus/0003-fix-ipython-display.patch'
Applying /etc/patches/smus/0003-fix-ipython-display.patch
+ cd /opt/conda
+ patch --strip=3
patching file lib/python3.12/site-packages/hdijupyterutils/ipythondisplay.py
+ for PATCHFILE in "$PATCH_DIR"/*.patch
+ '[' -f /etc/patches/smus/0004-fix-boto3-endpoints.patch ']'
+ should_skip_patch /etc/patches/smus/0004-fix-boto3-endpoints.patch
+ local patch_file=/etc/patches/smus/0004-fix-boto3-endpoints.patch
++ basename /etc/patches/smus/0004-fix-boto3-endpoints.patch
+ local patch_basename=0004-fix-boto3-endpoints.patch
+ [[ 0004-fix-boto3-endpoints.patch == *\f\i\x\-\i\p\y\t\h\o\n\-\d\i\s\p\l\a\y* ]]
+ [[ 0004-fix-boto3-endpoints.patch == *\f\i\x\-\b\o\t\o\3\-\e\n\d\p\o\i\n\t\s* ]]
++ get_package_version botocore
++ local package_name=botocore
+++ pip show botocore
+++ cut '-d ' -f2
+++ grep Version:
++ local pkg_version=1.37.1
++ '[' -z 1.37.1 ']'
++ echo 1.37.1
+ local botocore_version=1.37.1
+ '[' -n 1.37.1 ']'
+ version_gte 1.37.1 1.37.17
+ local version1=1.37.1
+ local version2=1.37.17
+ '[' -z 1.37.1 ']'
+ '[' -z 1.37.17 ']'
+ '[' 1.37.1 = 1.37.17 ']'
++ printf '%s\n%s\n' 1.37.1 1.37.17
++ sort -V
+ local 'sorted=1.37.1
1.37.17'
++ echo '1.37.1
1.37.17'
++ head -n1
+ local first_line=1.37.1
+ '[' 1.37.1 = 1.37.17 ']'
+ return 1
+ echo 'Applying /etc/patches/smus/0004-fix-boto3-endpoints.patch'
Applying /etc/patches/smus/0004-fix-boto3-endpoints.patch
+ cd /opt/conda
+ patch --strip=3
patching file lib/python3.12/site-packages/botocore/data/endpoints.json
Hunk #1 succeeded at 6561 (offset 1157 lines).

The offset occurred because the target file was slightly different from the file the patch was originally created for, but the patch command was still able to find the correct location to apply the changes by looking at the surrounding context.

Checklist:

• [ x ] My code follows the style guidelines of this project
• [ x ] I have commented my code, particularly in hard-to-understand areas
• [ x ] I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works

Test Screenshots (if applicable):

• Before Patches, unexpected session table persists

- After Patches, unexpected session table removed

Related Issues

Additional Notes

[AshleyZSY]

Patch 0003 can be removed if hdijupyterutils version is >= 0.23.0
https://github.com/jupyter-incubator/sparkmagic/blob/0.23.0/hdijupyterutils/hdijupyterutils/ipythondisplay.py

Please find the same for patch 004 and confirm with SMD team that future releases will not include any version lower than that.

Or let's add min version requirement.

[minshinz]

@AshleyZSY 2.6 or older, 3.1 or older are the ones has hdijupyterutils<0.23 . They should deprecating

so all non-deprecated versions should have hdijupyterutils 0.23

[minshinz]

Amended the commit because I have updated the apply_patches.sh file where version check for patch 3/4 is applied.

We need this version check because currently there are discrenpency exists in different SMD release. That is, some packages that are intended to be patched have upgraded. As a consequence, the patch is no longer needed and buggy.

With this version check, we can ensure patches are securely applied to corresponding SMD Versions.