POC demonstrating how to use guest attributes to improve notebook VM startup output. #450

mbookman · 2023-03-27T21:04:18Z

Note intended as-is for commit. Intended as POC.

A good implementation would probably include getting attributes through the GCP API (through a wrapper in the
terra-cloud-resource-lib).

Would anticipate having command-line flags for people to choose whether they want to fire and forget or wait for completion.

notebook VM startup output. A good implementation would probably include getting attributes through the GCP API (through a wrapper in the terra-cloud-resource-lib). Would anticipate having command-line flags for people to choose whether they want to fire and forget or wait for completion.

mbookman · 2023-03-27T21:14:25Z

Example output: SUCCESS

$ terra resource create gcp-notebook --name=test1
Successfully added controlled GCP Notebook instance.
Name:         test1
Description:
Type:         AI_NOTEBOOK
Stewardship:  CONTROLLED
Cloning:      COPY_RESOURCE
Access scope: PRIVATE_ACCESS
Managed by:   USER
Private user: test@test.com
Properties:   class Properties {
    []
}
GCP project id:                terra-hidden-7350
Instance id:   test1
Location: us-central1-a
Instance name: projects/terra-hidden-7350/locations/us-central1-a/instances/test1
State:         PROVISIONING
Metadata:
   warmup-libraries: matplotlib.pyplot
   proxy-mode: service_account
   terra-workspace-id: test-dev-workspace
   shutdown-script: /opt/deeplearning/bin/shutdown_script.sh
   post-startup-script: https://raw.githubusercontent.com/DataBiosphere/terra-workspace-manager/main/service/src/main/java/bio/terra/workspace/service/resource/controlled/cloud/gcp/ainotebook/post-startup.sh
   notebooks-api-version: v1
   terra-cli-server: verily-devel
   disable-swap-binaries: true
   enable-guest-attributes: TRUE
   serial-port-logging-enable: true
   notebooks-api: PROD
Proxy URL:     (undefined)
Create time:   2023-03-27T17:41:52.825217851Z
ERROR getting notebook startup status: 1
The post-startup script has not started.
ERROR getting notebook startup status: 1
The post-startup script has not started.
ERROR getting notebook startup status: 1
The post-startup script has not started.
ERROR getting notebook startup status: 1
The post-startup script has not started.
ERROR getting notebook startup status: 1
The post-startup script has not started.

STARTED..........
COMPLETE

Example output: FAILURE

$ terra resource create gcp-notebook --name=test2 --post-startup-script gs://test-bucket-terra-dev/post-startup.sh
Successfully added controlled GCP Notebook instance.
Name:         test2
Description:  
Type:         AI_NOTEBOOK
Stewardship:  CONTROLLED
Cloning:      COPY_RESOURCE
Access scope: PRIVATE_ACCESS
Managed by:   USER
Private user: test@test.com
Properties:   class Properties {
    []
}
GCP project id:                terra-hidden-7350
Instance id:   test2
Location: us-central1-a
Instance name: projects/terra-hidden-7350/locations/us-central1-a/instances/test2
State:         PROVISIONING
Metadata:
   terra-workspace-id: test-dev-workspace
   post-startup-script: gs://test-bucket-terra-dev/post-startup.sh
   warmup-libraries: matplotlib.pyplot
   notebooks-api-version: v1
   proxy-mode: service_account
   shutdown-script: /opt/deeplearning/bin/shutdown_script.sh
   disable-swap-binaries: true
   notebooks-api: PROD
   enable-guest-attributes: TRUE
   serial-port-logging-enable: true
   terra-cli-server: verily-devel
Proxy URL:     (undefined)
Create time:   2023-03-27T19:57:05.818802611Z
ERROR getting notebook startup status: 1
The post-startup script has not started.
ERROR getting notebook startup status: 1
The post-startup script has not started.
ERROR getting notebook startup status: 1
The post-startup script has not started.
ERROR getting notebook startup status: 1
The post-startup script has not started.
ERROR getting notebook startup status: 1
The post-startup script has not started.

STARTED......
ERROR
Error on line 1, command "exit". See /home/jupyter/.terra/post-startup-output.txt for more information.