Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions example-deployment/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ others:
For this example we will be using [rmf_demos](https://github.com/open-rmf/rmf_demos) as a local "deployment" of RMF, check that you have a working installation with

```bash
ros2 launch rmf_demos_gz office.launch.xml
RMW_IMPLEMENTATION=rmw_fastrtps_cpp ros2 launch rmf_demos_gz office.launch.xml
```

## kubernetes
Expand Down Expand Up @@ -276,9 +276,10 @@ The jobs we will run in this example can be found at `k8s/example-full/cronjobs.
## Test the deployment

If not done so already, launch the office demo
NOTE: Due to DDS challenging behavior with minikube over NAT, we will use `rmw_fastrtps_cpp` to bypass NAT traversal using sharede memory transport.

```bash
ros2 launch rmf_demos_gz office.launch.xml headless:=true
RMW_IMPLEMENTATION=rmw_fastrtps_cpp ros2 launch rmf_demos_gz office.launch.xml headless:=1 use_sim_time:=false server_uri:=ws://example.com:8001
```

Go to https://example.com/dashboard, if everything works, you should see a log in screen, use user=example, password=example.
Expand All @@ -292,7 +293,7 @@ It can be very useful to automate everything mentioned above and instantly deplo
Before you run the script, first you have to obtain the source. Refer to the above instructions if you don't know how to do it. You should have a git repo for rmf-web and a colcon workspace for rmf, then just run

```bash
./deploy.sh --rmf-ws <path-to-rmf-ws> --rmf-web-ws <path-to-rmf-web-ws>
./deploy.sh <path-to-rmf-ws> <path-to-rmf-web-ws>
```

Note: If you followed the above instructions to obtain the source, your rmf workspace will be in `ws/rmf`, and your rmf-web workspace will be in `ws/rmf-web`. So you will run it with `./deploy.sh --rmf-ws ws/rmf --rmf-web-ws ws/rmf`.
Expand Down Expand Up @@ -399,7 +400,11 @@ If you can connect to rmf, there is a chance that there is some discovery issues

## Some containers crashes at first startup

Some deployments like the rmf-server may crash on the first startup, this is often due to the database not being ready yet. This is normal and kubernetes will automatically restart the container.
Some deployments like the rmf-server may crash on the first startup, this is often due to the database not being ready yet. This is normal and kubernetes will automatically restart the container. Alternatively, you can forcefully terminate the rmf-server to force a restart:
```
kubectl get pods
kubectl delete pods [rmf-server-xxxx]
```

One solution is to add an init container to probe for the readiness of the database, but this comes at a small cost which is not normally expected to happen in an actual deployment. It is also not fail proof as there is no guarantee that

Expand Down
1 change: 0 additions & 1 deletion example-deployment/build-builder.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ cd $root_dir
mkdir -p build/builder/

cd $1
cp lerna.json "$root_dir/build/builder/"
find . -name node_modules -prune -o -name build -prune -o \
\(\
-name 'package*.json' -o \
Expand Down
2 changes: 1 addition & 1 deletion example-deployment/deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ echo '🗒 setting up apps and users'
try() {
"$@" || (sleep 1 && "$@") || (sleep 5 && "$@")
}
try node keycloak-tools/bootstrap-keycloak.js
try node keycloak-tools/bootstrap-keycloak.js || true # persistent keycloak database might fail this script if it has already been run before
try node keycloak-tools/get-cert.js > k8s/example-full/keycloak/keycloak.pem
openssl x509 -in k8s/example-full/keycloak/keycloak.pem -pubkey -noout -out k8s/example-full/keycloak/jwt-pub-key.pub
echo '✅ successfully setup keycloak'
Expand Down
2 changes: 0 additions & 2 deletions example-deployment/docker/dashboard.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@ ARG BUILDER_TAG
FROM rmf-web/builder:$BUILDER_TAG

COPY . /root/rmf-web
RUN cd /root/rmf-web && \
lerna run prepare --include-dependencies --scope=rmf-dashboard

ARG PUBLIC_URL
ARG REACT_APP_TRAJECTORY_SERVER
Expand Down
39 changes: 0 additions & 39 deletions example-deployment/docker/dashboard.dockerfile

This file was deleted.

1 change: 0 additions & 1 deletion example-deployment/docker/reporting-server.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ FROM rmf-web/builder:$BUILDER_TAG

COPY . /root/rmf-web
RUN cd /root/rmf-web && \
lerna run prepare --include-dependencies --scope=reporting-server && \
cd packages/reporting-server && \
npm run prepack

Expand Down
6 changes: 0 additions & 6 deletions example-deployment/docker/reporting.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@ ARG BUILDER_TAG
FROM rmf-web/builder:$BUILDER_TAG

COPY . /root/rmf-web
RUN cd /root/rmf-web && \
lerna run prepare --include-dependencies --scope=reporting

ARG PUBLIC_URL
ARG REACT_APP_REPORTING_SERVER
Expand Down Expand Up @@ -31,10 +29,6 @@ RUN echo -e 'server {\n\
index index.html index.htm;\n\
try_files $uri /reporting/index.html;\n\
}\n\
\n\
location /reporting/static/ {\n\
expires 30d;\n\
}\n\
\n\
error_page 500 502 503 504 /50x.html;\n\
location = /50x.html {\n\
Expand Down
7 changes: 4 additions & 3 deletions example-deployment/docker/rmf-server.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
FROM rmf-web/rmf-server:build as stage0

FROM ros:foxy-ros-base-focal
FROM ros:galactic

SHELL ["bash", "-c"]

Expand All @@ -15,7 +15,8 @@ COPY rmf/rmf_internal_msgs/rmf_traffic_msgs /root/rmf_ws/src/rmf_traffic_msgs
COPY rmf/rmf_internal_msgs/rmf_workcell_msgs /root/rmf_ws/src/rmf_workcell_msgs
COPY rmf/rmf_building_map_msgs /root/rmf_ws/src/rmf_building_map_msgs

RUN . /opt/ros/foxy/setup.bash && cd /root/rmf_ws && \
RUN apt update && apt install -y ros-galactic-rmw-fastrtps-cpp
RUN . /opt/ros/galactic/setup.bash && cd /root/rmf_ws && \
colcon build --merge-install --install-base /opt/rmf --cmake-args -DCMAKE_BUILD_TYPE=Release && \
rm -rf /root/rmf_ws

Expand All @@ -27,6 +28,6 @@ RUN cd /root/rmf-server && \

RUN echo -e '#!/bin/bash\n\
. /opt/rmf/setup.bash\n\
exec rmf_api_server "$@"\n\
rmf_api_server "$@"\n\
' > /docker-entry-point.sh && chmod +x /docker-entry-point.sh
ENTRYPOINT ["/docker-entry-point.sh"]
1 change: 0 additions & 1 deletion example-deployment/docker/rmf-server.build.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,5 @@ FROM rmf-web/builder:$BUILDER_TAG

COPY . /root/rmf-web
RUN cd /root/rmf-web && \
lerna run prepare --include-dependencies --scope=api-server && \
cd packages/api-server && \
npm run prepack
8 changes: 4 additions & 4 deletions example-deployment/k8s/base/reporting-server/fluentd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
---
# to set permission and groups
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
kind: ClusterRole
metadata:
name: fluentd
namespace: default
Expand All @@ -20,12 +20,12 @@ rules:
- list
- watch
---
kind: RoleBinding
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: fluentd
roleRef:
kind: Role
kind: ClusterRole
name: fluentd
apiGroup: rbac.authorization.k8s.io
subjects:
Expand Down Expand Up @@ -96,4 +96,4 @@ spec:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
path: /var/lib/docker/containers
19 changes: 19 additions & 0 deletions example-deployment/k8s/base/rmf-server/rmf-server.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,23 @@ spec:
ports:
- protocol: TCP
port: 8000

---
apiVersion: v1
kind: Service
metadata:
name: rmf-server-ws
labels:
app: rmf-server-ws
tier: app
spec:
selector:
app: rmf-server
tier: app
ports:
- protocol: TCP
port: 8001
Comment on lines +84 to +98
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not needed anymore as the api-server no longer uses multiple ports.


---
apiVersion: apps/v1
kind: Deployment
Expand Down Expand Up @@ -107,6 +124,8 @@ spec:
env:
- name: RMF_API_SERVER_CONFIG
value: /rmf-server-config/rmf_server_config.py
- name: RMW_IMPLEMENTATION
value: rmw_fastrtps_cpp
Comment on lines +127 to +128
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recalled that this was added to workaround minikube misconfiguration, iirc the problem was because minikube isolates the networks so other dds wouldn't work if rmf is ran locally and api-server is ran on the cluster. It only works for fastrtps because it supports shared memory.

I think we should remove this as abusing shared memory features to workaround broken network configuration is wrong. iirc it is not possible to create a configuration that works for other dds without also putting rmf inside the cluster, and even if it is in the cluster, it might still not work because dds requires udp broadcast which might not be supported by minikube.

We can also consider writing some deployment instructions for rmf + kubernetes in the rmf meta repo and link to that here since deploying rmf is out of scope of this repo. It might also make sense to move this example deployment to the meta repo as deployment of rmf + rmf-web has become quite coupled together, making it hard to have separate docs without explaining the other.

volumeMounts:
- mountPath: /rmf-server-config
name: rmf-server-config
Expand Down
2 changes: 1 addition & 1 deletion example-deployment/k8s/example-full/rmf_server_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

config = deepcopy(default_config)
config["host"] = "0.0.0.0"
config["port"] = 8000
config["base_port"] = 8000
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New api-server reverts to using port.

config["db_url"] = "postgres://rmf-server:rmf-server@rmf-server-db/rmf-server"
config["public_url"] = "https://example.com/rmf/api/v1"
config["log_level"] = "INFO"
Expand Down
5 changes: 5 additions & 0 deletions example-deployment/kustomize-env.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
#!/bin/bash
set -e

kubectl() {
.bin/minikube kubectl -- "$@"
}
export -f kubectl

kubectl kustomize "$1" | envsubst
2 changes: 1 addition & 1 deletion package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 5 additions & 1 deletion packages/reporting/src/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,14 @@ import React from 'react';
import ReactDOM from 'react-dom';
import App from './components/app';
import * as serviceWorker from './serviceWorker';
import LocalizationProvider from '@mui/lab/LocalizationProvider';
import AdapterDateFns from '@mui/lab/AdapterDateFns';
Comment on lines +5 to +6
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DateTimePicker has graduated from material lab and into "material-x". This will cause reporting to fail when we merge #615.


ReactDOM.render(
<React.StrictMode>
<App />
<LocalizationProvider dateAdapter={AdapterDateFns}>
<App />
</LocalizationProvider>
</React.StrictMode>,
document.getElementById('root'),
);
Expand Down