Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 50 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,7 @@ Thank you for your interest in contributing. We welcome issues and pull requests
## How to contribute

- **Bug reports and feature ideas:** Open an [issue](https://github.com/aukilabs/splatter-server/issues).
- **Code changes:** Open a pull request (PR) against `main`. Use a branch name that matches our CI (e.g. `feature/your-feature`, `bug/fix-description`, `chore/your-change`).

## Development setup

- **Rust (compute node):** See [server/rust/README.md](server/rust/README.md). Copy `server/rust/.env.example` to `server/rust/.env` and configure for local DDS/DMS if needed.
- **Docker:** From the repo root, `docker build -t splatter-server .` then run with `--env-file .env` as in the main [README](README.md).
- **Code changes:** Open a pull request (PR) against `develop`. Use a branch name that matches our CI (e.g. `feature/your-feature`, `fix/fix-description`, `chore/your-change`).

## Code standards

Expand All @@ -22,8 +17,56 @@ Thank you for your interest in contributing. We welcome issues and pull requests

## Pull request process

1. Point your PR at the `main` branch.
1. Point your PR at the `develop` branch.
2. Ensure CI passes (Rust format, clippy, tests).
3. Keep changes focused; link related issues where applicable.

By contributing, you agree that your contributions will be licensed under the same [MIT License](LICENSE) that covers this project.

# Local Development

## Development setup

- **Rust (compute node):** See [server/rust/README.md](server/rust/README.md). Copy `server/rust/.env.example` to `server/rust/.env` and configure for local DDS/DMS if needed.
- **Docker:** From the repo root, `docker build -t splatter-server .` then run with `--env-file .env`.

## Run Trainer
```bash
python3 run.py \
--domain_id {domain_id}
--job_id {job_id} \
--job_root_path {path/to/job/root} \
--log_level {log level}
```

## Input Files
```bash
# Input Files
{job_root_path}
├── datasets
│ └── {dataset}
│ └── Frames.mp4
├── refined
│ └── global
│ └── refined_sfm_combined
│ ├── cameras.bin
│ ├── images.bin
│ └── points3D.bin
```
## Output Files
```bash
# Output Files
{job_root_path}
├── Frames
│ ├── {images}
│ └── ...
├── refined
│ ├── nerfstudio-data
│ │ └── {converted nerfstudio data from colmap}
│ └── splatter
│ ├── splat.ply
│ ├── splat_rot.ply
│ ├── splat_rot.splat # this is what needs to be uploaded to dmt
│ └── splatfacto
│ └── {splat torch model}
```
73 changes: 17 additions & 56 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,14 @@
# splatter-server
# Splatter Node
This repository contains the splatter node, part of the Auki Network. This node enables photorealistic scene rendering by training 3D Gaussian Splats.

Splatter compute node for the Auki Network: runs Gaussian splatting jobs (COLMAP + nerfstudio/splatfacto) as part of the reconstruction pipeline.
The splatter node operates in conjunction with the [reconstruction node](https://github.com/aukilabs/reconstruction-server) and the scans from the Domain Management Tool (DMT) app ([App Store](https://apps.apple.com/app/domain-management-tool/id6499270503) 🔗). The refined camera poses from the reconstruction node are used as a starting point for training the gaussian splat, making it more robust to challenging indoor environments and noisy captures.

For more information about the reconstruction and rendering pipeline, please refer to our [whitepaper](https://auki.gitbook.io/whitepaper/technical-overview/the-reconstruction-service).

## Documentation
- [Minimum Requirements](docs/minimum-requirements.md)
- [Deployment](docs/deployment.md)
- [Contributing](CONTRIBUTING.md)

## License

Expand All @@ -10,59 +18,12 @@ This project is licensed under the [MIT License](LICENSE).

See [CONTRIBUTING.md](CONTRIBUTING.md) for how to report issues, open PRs, and run the Rust checks locally.

## Docker

### Build
From repo root path:
```bash
docker build -t splatter-server .
```

### Run

Create a `.env` from `server/rust/.env.example` and set your DMS/DDS URLs and registration secret, then:

```bash
docker run --gpus all -p 8080:8080 --name splatter -d --env-file .env splatter-server
```
## Acknowledgments

## Run Trainer
```bash
python3 run.py \
--domain_id {domain_id}
--job_id {job_id} \
--job_root_path {path/to/job/root} \
--log_level {log level}
```
This project builds upon the work of many excellent open-source projects, including
[nerfstudio](https://github.com/nerfstudio-project/nerfstudio), [ply2splat](https://github.com/bastikohn/ply2splat), [PyTorch](https://pytorch.org),
[OpenCV](https://opencv.org), [Open3D](https://www.open3d.org), and others.

## Required Files
```bash
# Input Files
{job_root_path}
├── datasets
│ └── {dataset}
│ └── Frames.mp4
├── refined
│ └── global
│ └── refined_sfm_combined
│ ├── cameras.bin
│ ├── images.bin
│ └── points3D.bin
```
## Output Files
```bash
# Output Files
{job_root_path}
├── Frames
│ ├── {images}
│ └── ...
├── refined
│ ├── nerfstudio-data
│ │ └── {converted nerfstudio data from colmap}
│ └── splatter
│ ├── splat.ply
│ ├── splat_rot.ply
│ ├── splat_rot.splat # this is what needs to be uploaded to dmt
│ └── splatfacto
│ └── {splat torch model}
```
We thank their authors and contributors for making this work possible.
Please note that all third-party code and libraries are subject to their respective licenses, copyrights, and trademarks.
We are not affiliated with, endorsed by, or sponsored by any of the projects or organizations mentioned above.
138 changes: 138 additions & 0 deletions docs/deployment.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# Deployment

The splatter node is available on [Docker Hub](https://hub.docker.com/r/aukilabs/splatter-server). Both deployment options are Docker-based and still run the same splatter logic under the hood.

## Initial Setup

1. First check that your NVIDIA driver and CUDA toolkit meet the requirements in the [Minimum Requirements](minimum-requirements.md) and update as needed:
```shell
nvidia-smi
```
This should output information about your GPU. If not, please double check that your installed CUDA and driver versions are correct, and restart your computer.

2. Ensure outbound HTTPS access to DDS/DMS endpoints from this host.

3. Disable power-saving settings like automatic sleep or standby mode, to keep your computer on and able to receive jobs.

## Prepare credentials + env file

Before you run the container, create a `.env` file based on the example file `server/rust/.env.example` with the required credentials:
```shell
REG_SECRET=your-registration-secret
SECP256K1_PRIVHEX=your-evm-private-key
```
Optional overrides (defaults shown):
```shell
DMS_BASE_URL=https://dms.auki.network/v1
DDS_BASE_URL=https://dds.auki.network
REQUEST_TIMEOUT_SECS=60
REGISTER_INTERVAL_SECS=120
REGISTER_MAX_RETRY=-1
LOG_FORMAT=text
```

Notes:
- `REG_SECRET` comes from registering a **compute node** in the Posemesh Console.
- `SECP256K1_PRIVHEX` is the hex-encoded private key of the **staked** EVM wallet for that node.

How to get the registration secret + wallet key:

⚠️ **IMPORTANT:** If you run both the reconstruction and splatter nodes, you need to stake them both, on different wallets.

1. Log in to the Posemesh Console at `https://console.auki.network/`.
2. Open the **Manage Nodes** page and create a compute node (choose the appropriate operation mode).
3. Copy the registration credentials shown — you will use these as `REG_SECRET`.
4. Go to **Staking**, connect your wallet, and stake the required amount of $AUKI for that node.
5. Export the wallet private key (hex) and set it as `SECP256K1_PRIVHEX`.

## Option 1 — Use the prebuilt image (recommended)

Pull the latest stable docker image. If you are upgrading from a previous version you must **run this again** to pull the updated image.
```shell
docker pull aukilabs/splatter-server:stable
```

Start Docker using the below command, ❗**including all flags**❗
```shell
docker run \
--gpus all \
--env-file .env \
--name splatter-server \
-d \
aukilabs/splatter-server:stable
```
You can also pin a specific release tag, for example `aukilabs/splatter-server:vX.Y.Z`.

### Verification ✅

1. After deploying, please ensure the server started correctly by running
```shell
docker ps
```
This should show your newly started docker container, with the STATUS showing `Up 45 seconds` or similar.
Copy the container ID of your server, then run:
```shell
docker logs <container_id>
```
You should see logs indicating the node is registering with DDS and polling DMS.

2. Ensure your GPU and CUDA works correctly (using the container ID from above):
```shell
docker exec <container_id> nvidia-smi
```
This should show your GPU and driver information.

Verify that torch detects your GPU:
```shell
docker exec <container_id> python3 -c "import torch; print(torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'CUDA not found')"
```

If not, please double-check your setup, or see **Troubleshooting**.

## Option 2 — Build Docker image from source

### Building Docker

```bash
docker build -t splatter-server .
```

Run the image as described in Option 1, and follow the same verification steps.

## Troubleshooting ⚠️

Here are some common issues you may encounter, with suggested fixes:

### GPU not detected
- **Symptom:** Server starts but jobs fail, GPU not utilised, `CUDA not found`, or similar.
- **Fixes:**
- Ensure you started the Docker container with `--gpus all`.
- Check your driver and CUDA toolkit versions against [Minimum Requirements](minimum-requirements.md).
- Run `nvidia-smi` on host to confirm your GPU is visible.
- Run `nvcc --version` to confirm your CUDA toolkit is installed, and with the correct version.
- Restart your computer and try again.

### Wrong GPU used
- **Symptom:** The Docker container is using the wrong GPU, for example the integrated GPU instead of the discrete GPU.
- **Fixes:**
- Check your system activity to see which GPU is being used.
- Run `nvidia-smi -L` to list all GPUs on your system.
- Run the container with the correct GPU index specified in the `--gpus` flag, for example `docker run --gpus '"device=0"' ...`.

### Container killed or crashes under load
- **Symptom:** Server stops or computer restarts during job processing
- **Fix:**
- Monitor system RAM and temperatures, and check for overheating or insufficient resources.

### Docker crashes on Windows
- **Symptom:** Container stops after a few minutes on Windows.
- **Fix:**
- Restart Docker Desktop.
- If the issue persists, also restart your computer.

---

💡 **Still stuck?**
If your issue remains, please:
1. Check `docker logs <container_id>` for error messages.
2. Share logs and system specs with the [Auki](https://www.auki.com) team for support.
17 changes: 17 additions & 0 deletions docs/minimum-requirements.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Minimum Requirements

The splatter node has fairly high requirements as the training of a realistic gaussian splat is computationally intensive. Please ensure your system is able to handle high CPU and GPU load for extended periods without overheating.

**NOTE:** The requirements below are high to ensure smooth processing. If you successfully run on lower hardware specs, please let us know!

- **OS (64-bit):** Windows 10 / 11 or Ubuntu 22.04 / 24.04 LTS
- **CPU:** 4 cores
- **RAM:** 12 GiB (recommended 16 GiB or more)
- **GPU:** Nvidia with 8+ GiB VRAM. Tested on RTX 3090, RTX 4060 and T4. RTX 50xx is not currently supported, but planned for upcoming releases. May work on older Nvidia cards too with enough VRAM and recent CUDA.
- **NVIDIA driver:** Recent driver with CUDA 11.8 support.
- **Disk space:** 40 GB or more
- **Docker** _- Windows support tested with Docker Desktop and WSL 2_
- Sufficient cooling and power supply to handle high load.
- A stable Internet connection with at least 10 Mbps downstream and upstream

See [Deployment](deployment.md) for more information.