Skip to content

feat(ws): setup virtualservice to route traffic for workspaces #500

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: notebooks-v2
Choose a base branch
from
Open

feat(ws): setup virtualservice to route traffic for workspaces #500

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion workspaces/backend/internal/models/workspaces/funcs.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -359,7 +359,7 @@ func buildServices(ws *kubefloworgv1beta1.Workspace, wskPodTemplatePorts map[kub
case kubefloworgv1beta1.ImagePortProtocolHTTP:
services[i].HttpService = &HttpService{
DisplayName: ptr.Deref(port.DisplayName, wskPort.DefaultDisplayName),
HttpPath: fmt.Sprintf("/workspace/%s/%s/%s/", ws.Namespace, ws.Name, port.Id),
HttpPath: fmt.Sprintf("/workspace/connect/%s/%s/%s/", ws.Namespace, ws.Name, port.Id),
Copy link
Member

@thesuperzapper thesuperzapper Sep 25, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, we should update all other instances which look like /workspace/... path string formatters (do a search over the whole project)

In addition, we should update the doc string of the WorkspaceKind types (in controller) and the doc string of the WorkspaceKind same (which should usually match the go doc string anyway).

}
}
}
Expand Down
4 changes: 2 additions & 2 deletions workspaces/controller/api/v1beta1/workspacekind_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,7 @@ type WorkspaceKindVolumeMounts struct {

type HTTPProxy struct {
// if the path prefix is stripped from incoming HTTP requests
// - if true, the '/workspace/{profile_name}/{workspace_name}/' path prefix
// - if true, the '/workspace/connect/{profile_name}/{workspace_name}/' path prefix
// is stripped from incoming requests, the application sees the request
// as if it was made to '/...'
// - this only works if the application serves RELATIVE URLs for its assets
Expand All @@ -291,7 +291,7 @@ type HTTPProxy struct {
// - sets the `spec.http[].headers.request` of the Istio VirtualService
// https://istio.io/latest/docs/reference/config/networking/virtual-service/#Headers-HeaderOperations
// - the following string templates are available:
// - `.PathPrefix`: the path prefix of the Workspace (e.g. '/workspace/{profile_name}/{workspace_name}/')
// - `.PathPrefix`: the path prefix of the Workspace (e.g. '/workspace/connect/{profile_name}/{workspace_name}/')
// +kubebuilder:validation:Optional
RequestHeaders *IstioHeaderOperations `json:"requestHeaders,omitempty"`
}
Expand Down
3 changes: 3 additions & 0 deletions workspaces/controller/cmd/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import (
// to ensure that exec-entrypoint and run can make use of them.
_ "k8s.io/client-go/plugin/pkg/client/auth"

istiov1 "istio.io/client-go/pkg/apis/networking/v1"
"k8s.io/apimachinery/pkg/runtime"
utilruntime "k8s.io/apimachinery/pkg/util/runtime"
clientgoscheme "k8s.io/client-go/kubernetes/scheme"
Expand All @@ -50,6 +51,8 @@ var (
func init() {
utilruntime.Must(clientgoscheme.AddToScheme(scheme))

utilruntime.Must(istiov1.AddToScheme(scheme))

utilruntime.Must(kubefloworgv1beta1.AddToScheme(scheme))
// +kubebuilder:scaffold:scheme
}
Expand Down
4 changes: 2 additions & 2 deletions workspaces/controller/config/crd/bases/kubeflow.org_workspacekinds.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3699,7 +3699,7 @@ spec:
default: false
description: |-
if the path prefix is stripped from incoming HTTP requests
- if true, the '/workspace/{profile_name}/{workspace_name}/' path prefix
- if true, the '/workspace/connect/{profile_name}/{workspace_name}/' path prefix
is stripped from incoming requests, the application sees the request
as if it was made to '/...'
- this only works if the application serves RELATIVE URLs for its assets
Expand All @@ -3710,7 +3710,7 @@ spec:
- sets the `spec.http[].headers.request` of the Istio VirtualService
https://istio.io/latest/docs/reference/config/networking/virtual-service/#Headers-HeaderOperations
- the following string templates are available:
- `.PathPrefix`: the path prefix of the Workspace (e.g. '/workspace/{profile_name}/{workspace_name}/')
- `.PathPrefix`: the path prefix of the Workspace (e.g. '/workspace/connect/{profile_name}/{workspace_name}/')
properties:
add:
additionalProperties:
Expand Down
6 changes: 6 additions & 0 deletions workspaces/controller/config/manager/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@ labels:
pairs:
app.kubernetes.io/component: controller-manager

configMapGenerator:
- envs:
- params.env
name: config
generatorOptions:
disableNameSuffixHash: true
images:
- name: workspaces-controller
newName: ghcr.io/kubeflow/notebooks/workspaces-controller
Expand Down
16 changes: 16 additions & 0 deletions workspaces/controller/config/manager/manager.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,22 @@ spec:
- --health-probe-bind-address=:8081
- --metrics-bind-address=0
image: workspaces-controller:latest
env:
- name: USE_ISTIO
valueFrom:
configMapKeyRef:
name: config
key: USE_ISTIO
- name: ISTIO_GATEWAY
valueFrom:
configMapKeyRef:
name: config
key: ISTIO_GATEWAY
- name: ISTIO_HOST
valueFrom:
configMapKeyRef:
name: config
key: ISTIO_HOST
imagePullPolicy: IfNotPresent
name: manager
securityContext:
Expand Down
4 changes: 4 additions & 0 deletions workspaces/controller/config/manager/params.env
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
USE_ISTIO=true
ISTIO_GATEWAY=kubeflow/kubeflow-gateway
ISTIO_HOST=*
CLUSTER_DOMAIN=cluster.local
4 changes: 2 additions & 2 deletions workspaces/controller/config/samples/jupyterlab_v1beta1_workspacekind.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ spec:
httpProxy:

## if the path prefix is stripped from incoming HTTP requests
## - if true, the '/workspace/{profile_name}/{workspace_name}/' path prefix
## - if true, the '/workspace/connect/{profile_name}/{workspace_name}/' path prefix
## is stripped from incoming requests, the application sees the request
## as if it was made to '/...'
## - this only works if the application serves RELATIVE URLs for its assets
Expand All @@ -164,7 +164,7 @@ spec:
## - sets the `spec.http[].headers.request` of the Istio VirtualService
## https://istio.io/latest/docs/reference/config/networking/virtual-service/#Headers-HeaderOperations
## - the following string templates are available:
## - `.PathPrefix`: the path prefix of the Workspace (e.g. '/workspace/{profile_name}/{workspace_name}/')
## - `.PathPrefix`: the path prefix of the Workspace (e.g. '/workspace/connect/{profile_name}/{workspace_name}/')
##
requestHeaders: {}
#set: { "X-RStudio-Root-Path": "{{ .PathPrefix }}" } # for RStudio
Expand Down
3 changes: 3 additions & 0 deletions workspaces/controller/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ require (
github.com/onsi/ginkgo/v2 v2.19.0
github.com/onsi/gomega v1.33.1
golang.org/x/time v0.3.0
istio.io/api v1.22.8
istio.io/client-go v1.22.8
k8s.io/api v0.31.0
k8s.io/apimachinery v0.31.0
k8s.io/client-go v0.31.0
Expand Down Expand Up @@ -59,6 +61,7 @@ require (
golang.org/x/text v0.16.0 // indirect
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240528184218-531527333157 // indirect
google.golang.org/protobuf v1.34.2 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
Expand Down
10 changes: 8 additions & 2 deletions workspaces/controller/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/emicklei/go-restful/v3 v3.11.0 h1:rAQeMHw1c7zTmncogyy8VvRZwtkmkZ4FxERmMY4rD+g=
github.com/emicklei/go-restful/v3 v3.11.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/evanphx/json-patch v0.5.2 h1:xVCHIVMUu1wtM/VkR9jVZ45N3FhZfYMMYGorLCR8P3k=
github.com/evanphx/json-patch v0.5.2/go.mod h1:ZWS5hhDbVDyob71nXKNL0+PWn6ToqBHMikGIFbs31qQ=
github.com/evanphx/json-patch v5.6.0+incompatible h1:jBYDEEiFBPxA0v50tFdvOzQQTCvpL6mnFh5mB2/l16U=
github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/evanphx/json-patch/v5 v5.9.0 h1:kcBlZQbplgElYIlo/n1hJbls2z/1awpXxpRi0/FOJfg=
github.com/evanphx/json-patch/v5 v5.9.0/go.mod h1:VNkHZ/282BpEyt/tObQO8s5CMPmYYq14uClGH4abBuQ=
github.com/fsnotify/fsnotify v1.7.0 h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA=
Expand Down Expand Up @@ -155,6 +155,8 @@ golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8T
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gomodules.xyz/jsonpatch/v2 v2.4.0 h1:Ci3iUJyx9UeRx7CeFN8ARgGbkESwJK+KB9lLcWxY/Zw=
gomodules.xyz/jsonpatch/v2 v2.4.0/go.mod h1:AH3dM2RI6uoBZxn3LVrfvJ3E0/9dG4cSrbuBJT4moAY=
google.golang.org/genproto/googleapis/api v0.0.0-20240528184218-531527333157 h1:7whR9kGa5LUwFtpLm2ArCEejtnxlGeLbAyjFY8sGNFw=
google.golang.org/genproto/googleapis/api v0.0.0-20240528184218-531527333157/go.mod h1:99sLkeliLXfdj2J75X3Ho+rrVCaJze0uwN7zDDkjPVU=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
Expand All @@ -170,6 +172,10 @@ gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
istio.io/api v1.22.8 h1:mhkaeFJ13WZ2d6pvL9+exNeQ9UB6HX7e6m+XwO9XoYY=
istio.io/api v1.22.8/go.mod h1:S3l8LWqNYS9yT+d4bH+jqzH2lMencPkW7SKM1Cu9EyM=
istio.io/client-go v1.22.8 h1:wojmt220jSbfhpRDsPiflj2nSFTBuYtZNiW9hqKeaWE=
istio.io/client-go v1.22.8/go.mod h1:noO8SoyMxLwni3w+yGK67aydi2klExjmiqnXyeRS/00=
k8s.io/api v0.31.0 h1:b9LiSjR2ym/SzTOlfMHm1tr7/21aD7fSkqgD/CVJBCo=
k8s.io/api v0.31.0/go.mod h1:0YiFF+JfFxMM6+1hQei8FY8M7s1Mth+z/q7eF1aJkTE=
k8s.io/apiextensions-apiserver v0.31.0 h1:fZgCVhGwsclj3qCw1buVXCV6khjRzKC5eCFt24kyLSk=
Expand Down
5 changes: 5 additions & 0 deletions workspaces/controller/internal/controller/suite_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,9 @@ import (
"sigs.k8s.io/controller-runtime/pkg/log/zap"

kubefloworgv1beta1 "github.com/kubeflow/notebooks/workspaces/controller/api/v1beta1"

istiov1 "istio.io/client-go/pkg/apis/networking/v1"

"github.com/kubeflow/notebooks/workspaces/controller/internal/helper"
// +kubebuilder:scaffold:imports
)
Expand Down Expand Up @@ -88,6 +91,8 @@ var _ = BeforeSuite(func() {
By("setting up the scheme")
err = kubefloworgv1beta1.AddToScheme(scheme.Scheme)
Expect(err).NotTo(HaveOccurred())
err = istiov1.AddToScheme(scheme.Scheme)
Expect(err).NotTo(HaveOccurred())

// +kubebuilder:scaffold:scheme

Expand Down
Loading
Loading