diff --git a/api-reference/authentication.mdx b/api-reference/authentication.mdx new file mode 100644 index 0000000..08a3d66 --- /dev/null +++ b/api-reference/authentication.mdx @@ -0,0 +1,97 @@ +--- +title: "Authentication" +sidebarTitle: "Authentication" +--- + +Every request to the Vast.ai API must include an API key. This page covers how to create keys, how to include them in requests, and key lifecycle details. + +## Create an API Key + +Generate a key from the [Keys page](https://cloud.vast.ai/cli/keys/) in the web console: + +1. Click **+New**. +2. Give the key a name (optional but recommended — e.g. "CI pipeline" or "notebook"). +3. Copy the key immediately — you'll only see it once. + + +You can also create keys programmatically via the API ([Create API Key](/api-reference/accounts/create-api-key)), CLI ([`vastai create api-key`](/cli/reference/create-api-key)), or SDK ([`vast.create_api_key()`](/sdk/python/reference/create-api-key)). + + +## Use an API Key + +Include your key as a Bearer token in the `Authorization` header: + + +```bash cURL +curl -s -H "Authorization: Bearer $VAST_API_KEY" \ + "https://console.vast.ai/api/v0/users/current/" +``` + +```python Python +import os +import requests + +headers = {"Authorization": f"Bearer {os.environ['VAST_API_KEY']}"} +resp = requests.get("https://console.vast.ai/api/v0/users/current/", headers=headers) +print(resp.json()) +``` + + +A common pattern is to store your key in an environment variable: + +```bash +export VAST_API_KEY="your-api-key-here" +``` + +This keeps the key out of your code and makes it easy to rotate. + + +If you get a `401 Unauthorized` or `403 Forbidden` response, double-check your API key. The most common causes are a typo, an expired key, or a scoped key that lacks the required permission for the endpoint you're calling. + + +## Verify Your Key + +A quick way to confirm your key works is to fetch your account info: + +```bash +curl -s -H "Authorization: Bearer $VAST_API_KEY" \ + "https://console.vast.ai/api/v0/users/current/" +``` + +A successful response includes your user ID, email, balance, and SSH key: + +```json +{ + "id": 123456, + "email": "you@example.com", + "credit": 25.00, + "ssh_key": "ssh-rsa AAAAB3..." +} +``` + +## Scoped Keys and Permissions + +By default, the web console creates a **full-access** key. For CI/CD pipelines, shared tooling, or team environments, you should create **scoped keys** that restrict access to only the permissions you need. + +For example, a key that can only read and manage instances (but cannot access billing): + +```json +{ + "api": { + "misc": {}, + "user_read": {}, + "instance_read": {}, + "instance_write": {} + } +} +``` + +See the [Permissions](/api-reference/permissions) page for the full list of permission categories, endpoint mappings, constraint syntax, and advanced examples. + +## Key Expiration + +API keys do not expire by default. You can revoke a key at any time from the [Keys page](https://cloud.vast.ai/cli/keys/) or by calling the [Delete API Key](/api-reference/accounts/delete-api-key) endpoint. + + +Treat your API key like a password. Do not commit keys to version control or share them in plaintext. If a key is compromised, revoke it immediately and create a new one. + diff --git a/api-reference/introduction.mdx b/api-reference/introduction.mdx index 26c7e84..087922d 100644 --- a/api-reference/introduction.mdx +++ b/api-reference/introduction.mdx @@ -1,9 +1,330 @@ --- -title: "API Introduction" +title: "API Hello World" +sidebarTitle: "Hello World" --- **The raw REST API is intended for advanced users only.** These endpoints offer maximum flexibility but require you to manage all aspects of integration yourself. Most users will have a significantly better experience using the [CLI](/cli/get-started) or the [Python SDK](/sdk/python/quickstart), which handle these details for you. **If you are not sure whether you need direct API access, you almost certainly don't** — start with the CLI or SDK instead. -Welcome to Vast.ai's API documentation. Our API allows you to programmatically manage GPU instances, handle machine operations, and automate your AI/ML workflow. Whether you're running individual GPU instances or managing a fleet of machines, our API provides comprehensive control over all Vast.ai platform features. \ No newline at end of file +The Vast.ai REST API gives you programmatic access to the entire platform — authentication, account management, billing, GPU search, instance lifecycle, templates, volumes, serverless endpoints, and team administration. Anything you can do in the web console, you can automate through the API. + +This guide walks through the core workflow: authenticate, search for a GPU, rent it, wait for it to boot, connect to it, and clean up. By the end you'll understand the API calls needed to manage instances without touching the web console. + +## Prerequisites + +- A Vast.ai account with credit (~$0.01–0.05, depending on test instance run time) +- `curl` installed +- `python3` with the `requests` library (for the Python examples) + +## 1. Get Your API Key + +Generate an API key from the [Keys page](https://cloud.vast.ai/cli/keys/) by clicking **+New**. Copy the key — you'll need it for your API calls, and you'll only see it once. + +Export it as an environment variable: + +```bash +export VAST_API_KEY="your-api-key-here" +``` + + +The console creates a full-access key by default. You can also create scoped API keys with limited permissions via `POST /api/v0/auth/apikeys/` — useful for CI/CD or shared tooling where you want to restrict access to read-only or instance-only operations. See the [Create API Key](/api-reference/accounts/create-api-key) endpoint for details. + + +## 2. Verify Authentication + +Confirm your key works by fetching your account info. This returns your user ID, email, balance, and SSH key — a quick way to verify what your key grants access to. + + +```bash cURL +curl -s -H "Authorization: Bearer $VAST_API_KEY" \ + "https://console.vast.ai/api/v0/users/current/" +``` + +```python Python +import os +import requests + +VAST_API_KEY = os.environ["VAST_API_KEY"] +BASE_URL = "https://console.vast.ai/api/v0" +headers = {"Authorization": f"Bearer {VAST_API_KEY}"} + +resp = requests.get(f"{BASE_URL}/users/current/", headers=headers) +print(resp.json()) +``` + + +```json +{ + "id": 123456, + "email": "you@example.com", + "credit": 25.00, + "ssh_key": "ssh-rsa AAAAB3..." +} +``` + +The `credit` field shows your available credit, and `ssh_key` is the public key that will be injected into new instances. + + +If you get a `401` or `403`, double-check your API key. + + +## 3. Search for GPUs + +Find available machines using the bundles endpoint. This query returns the top 5 on-demand RTX 4090s sorted by deep learning performance benchmarked per dollar: + + +```bash cURL +curl -s -H "Authorization: Bearer $VAST_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "verified": {"eq": true}, + "rentable": {"eq": true}, + "gpu_name": {"eq": "RTX 4090"}, + "num_gpus": {"eq": 1}, + "direct_port_count": {"gte": 1}, + "order": [["dlperf_per_dphtotal", "desc"]], + "type": "on-demand", + "limit": 5 + }' \ + "https://console.vast.ai/api/v0/bundles/" +``` + +```python Python +search_params = { + "verified": {"eq": True}, + "rentable": {"eq": True}, + "gpu_name": {"eq": "RTX 4090"}, + "num_gpus": {"eq": 1}, + "direct_port_count": {"gte": 1}, + "order": [["dlperf_per_dphtotal", "desc"]], + "type": "on-demand", + "limit": 5, +} + +resp = requests.post(f"{BASE_URL}/bundles/", headers=headers, json=search_params) +offers = resp.json()["offers"] +for offer in offers: + print(f"ID: {offer['id']} GPU: {offer['gpu_name']} $/hr: {offer['dph_total']}") +``` + + +Each parameter in the query above controls a different filter: + +| Parameter | Value | Meaning | +|-----------|-------|---------| +| `verified` | `{"eq": true}` | Only machines verified by Vast.ai (identity-checked hosts) | +| `rentable` | `{"eq": true}` | Only machines currently available to rent | +| `gpu_name` | `{"eq": "RTX 4090"}` | Filter to a specific GPU model | +| `num_gpus` | `{"eq": 1}` | Exactly 1 GPU per instance | +| `direct_port_count` | `{"gte": 1}` | At least 1 directly accessible port (needed for SSH) | +| `order` | `[["dlperf_per_dphtotal", "desc"]]` | Sort by deep learning performance per dollar, best value first | +| `type` | `"on-demand"` | On-demand pricing (vs. interruptible spot/bid) | +| `limit` | `5` | Return at most 5 results | + +The response contains an `offers` array. Note the `id` of the offer you want — you'll use it in the next step. If no offers are returned, try relaxing your filters (e.g. a different GPU model or removing `direct_port_count`). + + +See the [Search Offers](/api-reference/search/search-offers) reference for the full list of filter parameters and operators. + + +## 4. Create an Instance + +Rent the machine by sending a PUT request to the asks endpoint. Replace `OFFER_ID` with the `id` from step 3. + +The simplest approach is to create from a **template**, which bundles your image, startup commands, and launch settings into a reusable configuration: + +```bash +curl -s -H "Authorization: Bearer $VAST_API_KEY" \ + -H "Content-Type: application/json" \ + -X PUT \ + -d '{ + "template_hash_id": "YOUR_TEMPLATE_HASH", + "disk": 20, + "runtype": "ssh_direct" + }' \ + "https://console.vast.ai/api/v0/asks/OFFER_ID/" +``` + + +A template packages your Docker image, environment variables, and startup script so you don't repeat them on every create call. See the [Templates API guide](/api-reference/creating-and-using-templates-with-api) for creating and managing templates. + + +If you don't have a template yet, you can specify the image and startup commands directly: + + +```bash cURL +curl -s -H "Authorization: Bearer $VAST_API_KEY" \ + -H "Content-Type: application/json" \ + -X PUT \ + -d '{ + "image": "pytorch/pytorch:2.4.0-cuda12.4-cudnn9-runtime", + "disk": 20, + "onstart": "echo hello && nvidia-smi", + "runtype": "ssh_direct" + }' \ + "https://console.vast.ai/api/v0/asks/OFFER_ID/" +``` + +```python Python +offer_id = offers[0]["id"] # from step 3 + +create_params = { + "image": "pytorch/pytorch:2.4.0-cuda12.4-cudnn9-runtime", + "disk": 20, + "onstart": "echo hello && nvidia-smi", + "runtype": "ssh_direct", +} + +resp = requests.put( + f"{BASE_URL}/asks/{offer_id}/", headers=headers, json=create_params +) +result = resp.json() +print(result) +``` + + +```json +{ + "success": true, + "new_contract": 12345678, + "instance_api_key": "d15a..." +} +``` + +Save the `new_contract` value — this is your instance ID. The `instance_api_key` provides scoped access for that specific instance (e.g., querying `GET /api/v0/instances/{id}/` without your main API key). + + +Setting `"runtype": "ssh_direct"` gives you a direct SSH connection to the machine, which has lower latency than the default proxy SSH. Recommended for interactive work. + + +## 5. Wait Until Ready + +The instance needs time to pull the Docker image and boot. Poll the status endpoint until `actual_status` is `"running"`. Replace `INSTANCE_ID` with the `new_contract` value from step 4. + + +```bash cURL +curl -s -H "Authorization: Bearer $VAST_API_KEY" \ + "https://console.vast.ai/api/v0/instances/INSTANCE_ID/" +``` + +```python Python +import time + +instance_id = result["new_contract"] + +while True: + resp = requests.get(f"{BASE_URL}/instances/{instance_id}/", headers=headers) + instance = resp.json()["instances"] + if instance is None: + print("Status: provisioning") + time.sleep(10) + continue + status = instance.get("actual_status") + print(f"Status: {status}") + if status == "running": + break + time.sleep(10) + +print(f"SSH: ssh -p {instance['ssh_port']} root@{instance['ssh_host']}") +``` + + +Example response: + +```json +{ + "instances": { + "actual_status": "loading", + "ssh_host": "...", + "ssh_port": 12345 + } +} +``` + +The `actual_status` field progresses through these states: + +| `actual_status` | Meaning | +|-----------------|---------| +| `null` | Instance is being provisioned | +| `"loading"` | Docker image is downloading | +| `"running"` | Ready to use | + +Poll every 10 seconds. Boot time is typically 1–5 minutes depending on the Docker image size. You can also use the `onstart` script to send a callback when the instance is ready, instead of polling. + +This endpoint returns extensive instance details (hardware specs, pricing, networking, utilization metrics, and more). See the [Show Instance](/api-reference/instances/show-instance) endpoint for the full field reference. + +Once `actual_status` is `"running"`, you're ready to connect. Since we set `runtype` to `ssh_direct` in step 4, this is a direct connection to the machine. + +## 6. Connect via SSH + +Use the `ssh_host` and `ssh_port` from the status response to connect directly to your new instance: + +```bash +ssh root@SSH_HOST -p SSH_PORT +``` + +## 7. Copy Data + +With SSH access, you can transfer files using any standard tool — `scp`, `rsync`, `rclone`, etc. For example: + +```bash +# Upload to instance +rsync -avz -e "ssh -p SSH_PORT" ./data/ root@SSH_HOST:/workspace/data/ + +# Download from instance +rsync -avz -e "ssh -p SSH_PORT" root@SSH_HOST:/workspace/results/ ./results/ +``` + +For cloud storage syncing and instance-to-instance transfers, see the [data movement guide](/documentation/instances/storage/data-movement). + +## 8. Clean Up + +When you're done, destroy the instance to stop all billing. + +Alternatively, to pause an instance temporarily instead of destroying it, you can **stop** it. Stopping halts compute billing but disk storage charges continue. + +**Destroy** (removes everything): + + +```bash cURL +curl -s -H "Authorization: Bearer $VAST_API_KEY" \ + -X DELETE \ + "https://console.vast.ai/api/v0/instances/INSTANCE_ID/" +``` + +```python Python +resp = requests.delete(f"{BASE_URL}/instances/{instance_id}/", headers=headers) +print(resp.json()) +``` + + +**Stop** (pauses compute, disk charges continue): + + +```bash cURL +curl -s -H "Authorization: Bearer $VAST_API_KEY" \ + -H "Content-Type: application/json" \ + -X PUT \ + -d '{"state": "stopped"}' \ + "https://console.vast.ai/api/v0/instances/INSTANCE_ID/" +``` + +```python Python +resp = requests.put( + f"{BASE_URL}/instances/{instance_id}/", + headers=headers, + json={"state": "stopped"}, +) +print(resp.json()) +``` + + +Both return `{"success": true}`. + +## Next Steps + +You've now completed the full instance lifecycle through the API: authentication, search, creation, polling, and teardown. From here: + +- **SSH setup** — See the [SSH guide](/documentation/instances/connect/ssh) for key configuration and advanced connection options. +- **Use templates** — Avoid repeating image and config parameters on every create call. The [Templates API guide](/api-reference/creating-and-using-templates-with-api) covers creating, sharing, and launching from templates. diff --git a/api-reference/permissions.mdx b/api-reference/permissions.mdx new file mode 100644 index 0000000..1c08e4b --- /dev/null +++ b/api-reference/permissions.mdx @@ -0,0 +1,238 @@ +--- +title: "Permissions" +sidebarTitle: "Permissions" +--- + +Every API key has a set of permissions that control which endpoints it can access. This page is the comprehensive reference for permission categories, how they map to API routes, and how to build custom scoped keys. + +For an overview of API key creation and usage, see [Authentication](/api-reference/authentication). + +## Permission Categories + +Permissions are organized into categories. When you create a scoped API key, you include only the categories the key needs. The available categories are: + +| Category | Controls | +|----------|----------| +| `instance_read` | Viewing instances, logs, SSH keys, volumes, deposits | +| `instance_write` | Creating, managing, and destroying instances and volumes | +| `user_read` | Viewing account info, API keys, SSH keys, environment variables, templates | +| `user_write` | Creating/modifying API keys, SSH keys, environment variables, templates, teams | +| `billing_read` | Viewing invoices and earnings | +| `billing_write` | Transferring credit | +| `machine_read` | Viewing machines and reports (hosts) | +| `machine_write` | Managing machines, maintenance, listing/unlisting (hosts) | +| `misc` | Search offers, benchmarks, network volumes, serverless endpoints | +| `team_read` | Viewing team members and roles | +| `team_write` | Inviting/removing team members, managing roles | + +## Creating Scoped Keys + +Define permissions as a JSON object and pass it when creating a key. The top-level key is always `"api"`, containing the categories you want to grant. + +**Example — Instance management with billing access:** + +```json +{ + "api": { + "misc": {}, + "user_read": {}, + "instance_read": {}, + "instance_write": {}, + "billing_read": {}, + "billing_write": {} + } +} +``` + +**Example — Instance management without billing:** + +```json +{ + "api": { + "misc": {}, + "user_read": {}, + "instance_read": {}, + "instance_write": {} + } +} +``` + +You can create scoped keys using: +- **API**: [Create API Key](/api-reference/accounts/create-api-key) +- **CLI**: [`vastai create api-key`](/cli/reference/create-api-key) +- **SDK**: [`vast.create_api_key()`](/sdk/python/reference/create-api-key) + +## Custom Roles + +Custom roles let you assign the same set of permissions to multiple team members. + +- **Creating roles**: Use the CLI or the Manage page in the web console (requires `team_write` access). +- **Defining permissions**: Select from any combination of the categories listed above. +- **Assigning roles**: Assign created roles to team members through the team management interface or CLI. + +## Constraints + +Constraints narrow a permission category to specific parameter values. This lets you create keys that can only operate on certain resources. + +**Example — Read logs for a single instance only:** + +```json +{ + "api": { + "instance_read": { + "api.instance.request_logs": { + "constraints": { + "id": { + "eq": 1227 + } + } + } + } + } +} +``` + +**Example — Read logs for a range of instance IDs:** + +```json +{ + "api": { + "instance_read": { + "api.instance.request_logs": { + "constraints": { + "id": { + "lte": 2, + "gte": 1 + } + } + } + } + } +} +``` + +Supported constraint operators: `eq`, `lte`, `gte`. + + +API keys using constraints must be created via the CLI ([`vastai create api-key`](/cli/reference/create-api-key)) or the API ([Create API Key](/api-reference/accounts/create-api-key)). + + +You can also use **wildcards** in `params` to represent placeholder values — useful when generating many keys that perform similar operations. + +## Endpoint Reference by Category + +Below is the complete mapping of which endpoints each permission category controls. + +### instance\_read + +- [Show Instance](/api-reference/instances/show-instance) +- [Show Instances](/api-reference/instances/show-instances) +- [Show Logs](/api-reference/instances/show-logs) +- [Show SSH Keys](/api-reference/instances/show-ssh-keys) +- [Show Volumes](/api-reference/volumes/list-volumes) +- [Show Deposit](/api-reference/billing/show-deposit) + +### instance\_write + +- [Attach SSH Key](/api-reference/instances/attach-ssh-key) +- [Copy](/api-reference/instances/copy) +- [Cancel Copy](/api-reference/instances/cancel-copy) +- [Cloud Copy](/api-reference/instances/cloud-copy) +- [Cancel Sync](/api-reference/instances/cancel-sync) +- [Change Bid](/api-reference/instances/change-bid) +- [Create Instance](/api-reference/instances/create-instance) +- [Manage Instance](/api-reference/instances/manage-instance) +- [Delete Instance](/api-reference/instances/destroy-instance) +- [Detach SSH Key](/api-reference/instances/detach-ssh-key) +- [Execute](/api-reference/instances/execute) +- [Prepay Instance](/api-reference/instances/prepay-instance) +- [Reboot Instance](/api-reference/instances/reboot-instance) +- [Recycle Instance](/api-reference/instances/recycle-instance) +- [Create Volume](/api-reference/volumes/rent-volume) +- [Delete Volume](/api-reference/volumes/delete-volume) + +### user\_read + +- [Show API Keys](/api-reference/accounts/show-api-keys) +- [Show Connections](/api-reference/accounts/show-connections) +- [Show Environment Variables](/api-reference/accounts/show-env-vars) +- [Show IP Addresses](/api-reference/accounts/show-ipaddrs) +- [Show SSH Keys](/api-reference/accounts/show-ssh-keys) +- [Show Subaccounts](/api-reference/accounts/show-subaccounts) +- [Show User](/api-reference/accounts/show-user) +- [Search Templates](/api-reference/search/search-template) + +### user\_write + +- [Create API Key](/api-reference/accounts/create-api-key) +- [Delete API Key](/api-reference/accounts/delete-api-key) +- [Create Environment Variable](/api-reference/accounts/create-env-var) +- [Update Environment Variable](/api-reference/accounts/update-env-var) +- [Delete Environment Variable](/api-reference/accounts/delete-env-var) +- [Create SSH Key](/api-reference/accounts/create-ssh-key) +- [Update SSH Key](/api-reference/accounts/update-ssh-key) +- [Delete SSH Key](/api-reference/accounts/delete-ssh-key) +- [Create Subaccount](/api-reference/accounts/create-subaccount) +- [Set User](/api-reference/accounts/set-user) +- [Create Team](/api-reference/team/create-team) +- [Delete Team](/api-reference/team/destroy-team) +- [Create Template](/api-reference/templates/create-template) +- [Edit Template](/api-reference/templates/edit-template) +- [Delete Template](/api-reference/templates/delete-template) + +### billing\_read + +- [Search Invoices](/api-reference/billing/search-invoices) +- [Show Invoices](/api-reference/billing/show-invoices) +- [Show Earnings](/api-reference/billing/show-earnings) + +### billing\_write + +- [Transfer Credit](/api-reference/accounts/transfer-credit) + +### machine\_read + +- [Show Machines](/api-reference/machines/show-machines) +- [Show Reports](/api-reference/machines/show-reports) + +### machine\_write + +- [Cancel Maintenance](/api-reference/machines/cancel-maint) +- [Cleanup Machine](/api-reference/machines/cleanup-machine) +- [List Machine](/api-reference/machines/list-machine) +- [Remove Default Job](/api-reference/machines/remove-defjob) +- [Schedule Maintenance](/api-reference/machines/schedule-maint) +- [Set Default Job](/api-reference/machines/set-defjob) +- [Set Minimum Bid](/api-reference/machines/set-min-bid) +- [Unlist Machine](/api-reference/machines/unlist-machine) +- [Add Network Disk](/api-reference/network-volumes/add-network-disk) +- [Unlist Network Volume](/api-reference/network-volumes/unlist-network-volume) +- [Unlist Volume](/api-reference/volumes/unlist-volume) + +### misc + +- [Search Network Volumes](/api-reference/network-volumes/search-network-volumes) +- [Show Workergroups](/api-reference/serverless/show-workergroup) +- [Create Workergroup](/api-reference/serverless/create-workergroup) +- [Update Workergroup](/api-reference/serverless/update-workergroup) +- [Delete Workergroup](/api-reference/serverless/delete-workergroup) +- [Show Endpoints](/api-reference/serverless/show-endpoints) +- [Create Endpoint](/api-reference/serverless/create-endpoint) +- [Delete Endpoint](/api-reference/serverless/delete-endpoint) +- [Search Benchmarks](/api-reference/search/search-benchmarks) +- [Search Offers](/api-reference/search/search-offers) +- [Search Volumes](/api-reference/volumes/search-volumes) + +### team\_read + +- [Show Team Members](/api-reference/team/show-team-members) +- [Show Team Role](/api-reference/team/show-team-role) +- [Show Team Roles](/api-reference/team/show-team-roles) + +### team\_write + +- [Invite Team Member](/api-reference/team/invite-team-member) +- [Remove Team Member](/api-reference/team/remove-team-member) +- [Create Team Role](/api-reference/team/create-team-role) +- [Update Team Role](/api-reference/team/update-team-role) +- [Remove Team Role](/api-reference/team/remove-team-role) diff --git a/cli/authentication.mdx b/cli/authentication.mdx new file mode 100644 index 0000000..c3267a0 --- /dev/null +++ b/cli/authentication.mdx @@ -0,0 +1,89 @@ +--- +title: "CLI Authentication" +sidebarTitle: "Authentication" +--- + +Every request to the Vast.ai API requires an API key. The CLI stores your key locally and includes it automatically in every command. This page covers how to set up, verify, and manage API keys through the CLI. + +## Set Your API Key + +After creating a key from the [Keys page](https://cloud.vast.ai/cli/keys/), store it locally: + +```bash +vastai set api-key YOUR_API_KEY +``` + +This writes the key to `~/.config/vastai/vast_api_key`. All subsequent commands use it automatically. + + +You can also create keys programmatically via the API ([Create API Key](/api-reference/accounts/create-api-key)) or SDK ([`vast.create_api_key()`](/sdk/python/reference/create-api-key)). + + +## Verify Your Key + +Confirm your key works by fetching your account info: + +```bash +vastai show user +``` + +A successful response includes your user ID, email, and balance: + +```json +{ + "id": 123456, + "email": "you@example.com", + "credit": 25.00, + "ssh_key": "ssh-rsa AAAAB3..." +} +``` + + +If you get an authentication error, double-check your API key. The most common causes are a typo, an expired key, or a scoped key that lacks the required permission for the command you're running. + + +## Create an API Key + +You can create new keys from the CLI: + +```bash +vastai create api-key --name "ci-deploy-key" +``` + +The output includes the new key value. Copy it immediately -- you will not be able to retrieve it again. + +To create a key with restricted permissions, pass a JSON permissions file: + +```bash +vastai create api-key --name "ci-deploy-key" --permission_file perms.json +``` + +See the [Permissions](/cli/permissions) page for the full permissions file format and examples. + +## View and Delete Keys + +List all API keys on your account: + +```bash +vastai show api-keys +``` + +View a specific key's details by ID: + +```bash +vastai show api-key 42 +``` + +Delete a key: + +```bash +vastai delete api-key 42 +``` + +## Key Expiration + +API keys do not expire by default. You can revoke a key at any time from the [Keys page](https://cloud.vast.ai/cli/keys/) or with `vastai delete api-key`. + + +Treat your API key like a password. Do not commit keys to version control or share them in plaintext. If a key is compromised, revoke it immediately and create a new one. + diff --git a/cli/hello-world.mdx b/cli/hello-world.mdx new file mode 100644 index 0000000..5d72b2f --- /dev/null +++ b/cli/hello-world.mdx @@ -0,0 +1,192 @@ +--- +title: "CLI Hello World" +sidebarTitle: "Hello World" +--- + +The Vast.ai CLI gives you command-line access to the entire platform — authentication, GPU search, instance lifecycle, templates, volumes, serverless endpoints, and more. Anything you can do in the web console, you can automate from your terminal. + +This guide walks through the core workflow: install the CLI, authenticate, search for a GPU, rent it, wait for it to boot, connect to it, copy data, and clean up. By the end you'll understand the commands needed to manage instances without touching the web console. + +## Prerequisites + +- A Vast.ai account with credit (~$0.01–0.05, depending on test instance run time) +- Python 3 installed + +## 1. Install the CLI + +Install from PyPI: + +```bash +pip install vastai +``` + +Or grab the latest version directly from GitHub: + +```bash +wget https://raw.githubusercontent.com/vast-ai/vast-python/master/vast.py -O vast && chmod +x vast +``` + +Verify the installation: + +```bash +vastai --help +``` + +## 2. Set Your API Key + +Generate an API key from the [Keys page](https://cloud.vast.ai/cli/keys/) by clicking **+New**. Copy the key — you'll only see it once. + +Save it to the CLI: + +```bash +vastai set api-key YOUR_API_KEY_HERE +``` + +This stores your key in a config file in your home directory. Do not share your API keys with anyone. + + +The console creates a full-access key by default. You can also create scoped keys with limited permissions using `vastai create api-key` — useful for CI/CD or shared tooling. See the [permissions documentation](/api-reference/permissions-and-authorization) for details. + + +## 3. Verify Authentication + +Confirm your key works by fetching your account info: + +```bash +vastai show user +``` + +This returns your user ID, email, balance, and SSH key. If you see an authentication error, double-check your API key. + +## 4. Search for GPUs + +Find available machines using `search offers`. This query returns on-demand RTX 4090s on verified machines with direct port access, sorted by deep learning performance per dollar: + +```bash +vastai search offers 'gpu_name=RTX_4090 num_gpus=1 verified=true direct_port_count>=1 rentable=true' -o 'dlperf_usd-' +``` + +Each parameter in the query controls a different filter: + +| Parameter | Meaning | +|-----------|---------| +| `gpu_name=RTX_4090` | Filter to a specific GPU model | +| `num_gpus=1` | Exactly 1 GPU per instance | +| `verified=true` | Only machines verified by Vast.ai (identity-checked hosts) | +| `direct_port_count>=1` | At least 1 directly accessible port (needed for direct SSH) | +| `rentable=true` | Only machines currently available to rent | +| `-o 'dlperf_usd-'` | Sort by DL performance per dollar, best value first | + +Note the `ID` of the offer you want — you'll use it in the next step. If no offers are returned, try relaxing your filters (e.g. a different GPU model or removing `direct_port_count`). + + +Use `vastai search offers --help` for the full list of filter fields and options, or see the [CLI commands reference](/cli/commands#search-offers). + + +## 5. Create an Instance + +Rent the machine using `create instance` with the offer ID from step 4: + +```bash +vastai create instance OFFER_ID --image pytorch/pytorch:2.4.0-cuda12.4-cudnn9-runtime --disk 20 --onstart-cmd "echo hello && nvidia-smi" --ssh --direct +``` + +| Flag | Meaning | +|------|---------| +| `--image` | Docker image to launch | +| `--disk 20` | 20 GB of disk storage | +| `--onstart-cmd` | Command to run when the instance boots | +| `--ssh --direct` | Direct SSH access (lower latency than proxy SSH) | + +The output includes the new instance ID: + +```json +{"success": true, "new_contract": 12345678} +``` + +Save the `new_contract` value — this is your instance ID. + + +Storage charges begin at creation. GPU charges begin when the instance reaches the `running` state. + + +## 6. Wait Until Ready + +The instance needs time to pull the Docker image and boot. Check the status with: + +```bash +vastai show instance INSTANCE_ID +``` + +The `status` field progresses through these states: + +| Status | Meaning | +|--------|---------| +| `loading` | Docker image is downloading | +| `running` | Ready to use | + +Check every 10–30 seconds. Boot time is typically 1–5 minutes depending on the Docker image size. + +## 7. Connect via SSH + +Once the instance is running, get the SSH connection details: + +```bash +vastai ssh-url INSTANCE_ID +``` + +Then connect: + +```bash +ssh root@SSH_HOST -p SSH_PORT +``` + +## 8. Copy Data + +Use `vastai copy` to transfer files between your local machine and the instance: + +```bash +# Upload to instance +vastai copy local:./data/ INSTANCE_ID:/workspace/data/ + +# Download from instance +vastai copy INSTANCE_ID:/workspace/results/ local:./results/ +``` + +You can also copy between instances or to/from cloud storage: + +```bash +# Instance to instance +vastai copy INSTANCE_A:/workspace/ INSTANCE_B:/workspace/ + +# Cloud storage (requires a configured cloud connection) +vastai copy s3.CONNECTION_ID:/bucket/data/ INSTANCE_ID:/workspace/ +``` + +For cloud storage syncing and instance-to-instance transfers, see the [data movement guide](/documentation/instances/storage/data-movement). + +## 9. Clean Up + +When you're done, destroy the instance to stop all billing. + +Alternatively, to pause an instance temporarily instead of destroying it, you can **stop** it. Stopping halts compute billing but disk storage charges continue. + +**Destroy** (removes everything): + +```bash +vastai destroy instance INSTANCE_ID +``` + +**Stop** (pauses compute, disk charges continue): + +```bash +vastai stop instance INSTANCE_ID +``` + +## Next Steps + +You've now completed the full instance lifecycle through the CLI: installation, authentication, search, creation, polling, data transfer, and teardown. From here: + +- **SSH setup** — See the [SSH guide](/documentation/instances/connect/ssh) for key configuration and advanced connection options. +- **Full command reference** — See the [CLI commands](/cli/commands) page for every available command. +- **Use templates** — Avoid repeating image and config parameters on every create call. See the [templates guide](/documentation/templates/introduction) for creating and managing templates. diff --git a/cli/permissions.mdx b/cli/permissions.mdx new file mode 100644 index 0000000..54337a2 --- /dev/null +++ b/cli/permissions.mdx @@ -0,0 +1,222 @@ +--- +title: "CLI Permissions" +sidebarTitle: "Permissions" +--- + +Every API key has a set of permissions that control which endpoints it can access. This page covers permission categories, how to build scoped keys, and how to manage team roles through the CLI. + +For an overview of API key creation and setup, see [Authentication](/cli/authentication). + +## Permission Categories + +Permissions are organized into categories. When you create a scoped API key, you include only the categories the key needs: + +| Category | Controls | +|----------|----------| +| `instance_read` | Viewing instances, logs, SSH keys, volumes, deposits | +| `instance_write` | Creating, managing, and destroying instances and volumes | +| `user_read` | Viewing account info, API keys, SSH keys, environment variables, templates | +| `user_write` | Creating/modifying API keys, SSH keys, environment variables, templates, teams | +| `billing_read` | Viewing invoices and earnings | +| `billing_write` | Transferring credit | +| `machine_read` | Viewing machines and reports (hosts) | +| `machine_write` | Managing machines, maintenance, listing/unlisting (hosts) | +| `misc` | Search offers, benchmarks, network volumes, serverless endpoints | +| `team_read` | Viewing team members and roles | +| `team_write` | Inviting/removing team members, managing roles | + +For the complete mapping of which specific endpoints each category controls, see [Permissions (API)](/api-reference/permissions#endpoint-reference-by-category). + +## Creating Scoped Keys + +Define permissions as a JSON file. The top-level key is always `"api"`, containing the categories you want to grant: + +```json +{ + "api": { + "misc": {}, + "user_read": {}, + "instance_read": {}, + "instance_write": {} + } +} +``` + +Save this as `perms.json`, then pass it to the CLI: + +```bash +vastai create api-key --name "ci-deploy-key" --permission_file perms.json +``` + +## Constraints + +Constraints narrow a permission category to specific parameter values. This lets you create keys that can only operate on certain resources. + +### Constrain by Exact ID + +This permissions file allows reading logs for instance 1227 only: + +```json +{ + "api": { + "instance_read": { + "api.instance.request_logs": { + "constraints": { + "id": { + "eq": 1227 + } + } + } + } + } +} +``` + +### Constrain by Range + +You can combine `gte` (greater than or equal) and `lte` (less than or equal) operators to define a range: + +```json +{ + "api": { + "instance_read": { + "api.instance.request_logs": { + "constraints": { + "id": { + "gte": 1, + "lte": 100 + } + } + } + } + } +} +``` + +Available constraint operators: `eq`, `gte`, `lte`. + + +Keys with constraints must be created through the CLI or API. The web console only creates full-access keys. + + +## Managing Team Roles + +Team roles use the same permission model as API keys. You define permissions in a JSON file and pass it to the team role commands. + +### Create a Role + +```bash +vastai create team-role --name "developer" --permissions perms.json +``` + +### View Roles + +List all roles for your team: + +```bash +vastai show team-roles +``` + +View a specific role by name: + +```bash +vastai show team-role developer +``` + +### Update a Role + +```bash +vastai update team-role 5 --name "senior-dev" --permissions updated-perms.json +``` + +### Remove a Role + +```bash +vastai remove team-role developer +``` + +### Invite a Team Member + +Assign a role when inviting a new member: + +```bash +vastai invite member --email teammate@example.com --role developer +``` + +### View Team Members + +```bash +vastai show members +``` + +## Examples + +### Read-Only Key + +A key that can view instances and account info but cannot create, modify, or destroy anything: + +```json +{ + "api": { + "instance_read": {}, + "user_read": {} + } +} +``` + +```bash +vastai create api-key --name "monitoring" --permission_file readonly.json +``` + +### Instance Management Without Billing + +A key that can create and manage instances but has no access to billing or credit transfers: + +```json +{ + "api": { + "misc": {}, + "user_read": {}, + "instance_read": {}, + "instance_write": {} + } +} +``` + +```bash +vastai create api-key --name "ci-deploy" --permission_file deploy.json +``` + +### Constrained Key for a Specific Instance + +A key that can only manage a single instance (view, reboot, destroy) and nothing else: + +```json +{ + "api": { + "instance_read": { + "api.instance.show": { + "constraints": { + "id": { "eq": 1227 } + } + } + }, + "instance_write": { + "api.instance.destroy": { + "constraints": { + "id": { "eq": 1227 } + } + }, + "api.instance.reboot": { + "constraints": { + "id": { "eq": 1227 } + } + } + } + } +} +``` + +```bash +vastai create api-key --name "instance-1227-only" --permission_file constrained.json +``` diff --git a/cli/rate-limits.mdx b/cli/rate-limits.mdx new file mode 100644 index 0000000..51a6587 --- /dev/null +++ b/cli/rate-limits.mdx @@ -0,0 +1,84 @@ +--- +title: "CLI Rate Limits and Errors" +sidebarTitle: "Rate Limits" +--- + +The Vast.ai CLI automatically retries rate-limited requests. You do not need to implement your own retry logic -- the CLI handles HTTP 429 responses with exponential backoff out of the box. + +This page covers the error format, how rate limits work, and how to configure the CLI's built-in retry behavior. For the full details on rate limit mechanics, see the [API Rate Limits and Errors](/api-reference/rate-limits-and-errors) page. + +## Error Responses + +When a CLI command fails, it prints the error message from the API and exits with a non-zero status code. The underlying API error shape is: + +```json +{ + "success": false, + "error": "invalid_args", + "msg": "Human-readable description of the problem." +} +``` + +Some endpoints omit `success` or `error` and return only `msg` or `message`. The CLI surfaces whatever message the API returns. + +## How Rate Limits Work + +Vast.ai applies rate limits **per endpoint** and **per identity**. The identity is determined by your bearer token, session user, `api_key` parameter, and client IP. + +Some endpoints also enforce **method-specific** limits (GET vs POST) and **max-calls-per-period** limits for short bursts. + +For the full breakdown, see [How rate limits are applied](/api-reference/rate-limits-and-errors#how-rate-limits-are-applied). + +## Rate Limit Response + +When you hit a rate limit, the API returns **HTTP 429** with a message like: + +``` +API requests too frequent +``` + +or + +``` +API requests too frequent: endpoint threshold=... +``` + +The API does not return a `Retry-After` header. The CLI handles this automatically using its built-in retry logic. + +## Built-in Retry Behavior + +When the CLI receives an HTTP 429 response, it automatically retries the request using exponential backoff: + +- **Retried status codes:** 429 only +- **Default retries:** 3 +- **Backoff strategy:** starts at 0.15 seconds, multiplied by 1.5x after each attempt +- **Retry delays:** ~0.15s, ~0.225s, ~0.34s + +For most usage, the defaults handle transient rate limits without any intervention. + + +The CLI only retries on 429 (rate limit). Other HTTP errors (4xx, 5xx) are reported immediately. + + +## Configuring Retries + +Use the `--retry` flag on any command to change the number of retry attempts: + +```bash +# Use default 3 retries +vastai search offers 'gpu_name=RTX_4090 rentable=true' + +# Increase to 6 retries for a batch script +vastai search offers 'gpu_name=RTX_4090 rentable=true' --retry 6 + +# Disable retries entirely +vastai search offers 'gpu_name=RTX_4090 rentable=true' --retry 0 +``` + +Increasing the retry count is useful for scripts that make many API calls in sequence, where occasional rate limiting is expected. + +## Reducing Rate Limit Errors + +If you are consistently hitting rate limits, the best approach is to reduce the volume and frequency of your requests. See [How to reduce rate limit errors](/api-reference/rate-limits-and-errors#how-to-reduce-rate-limit-errors) for practical strategies including batching, reduced polling, and traffic spreading. + +If you need higher limits for production usage, contact support with the endpoint(s), your expected call rate, and your account details. diff --git a/cli/reference/add-network-disk.mdx b/cli/reference/add-network-disk.mdx new file mode 100644 index 0000000..aeca3f4 --- /dev/null +++ b/cli/reference/add-network-disk.mdx @@ -0,0 +1,56 @@ +--- +title: "vastai add network-disk" +sidebarTitle: "add network-disk" +description: "Host command" +--- + +Add Network Disk to Physical Cluster. + +This is a **host** command, used for managing machines you are renting out on Vast.ai. + +## Usage + +```bash +vastai add network-disk MACHINES MOUNT_PATH [options] +``` + +## Arguments + + + ids of machines to add disk to, that is networked to be on the same LAN as machine + + + + mount path of disk to add + + +## Options + + + id of network disk to attach to machines in the cluster (alias: `--disk_id`) + + +## Description + +This variant can be used to add a network disk to a physical cluster. +When you add a network disk for the first time, you just need to specify the machine(s) and mount_path. +When you add a network disk for the second time, you need to specify the disk_id. + +## Examples + +```bash +vastai add network-disk 1 /mnt/disk1 +vastai add network-disk 1 /mnt/disk1 -d 12345 +``` + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/attach-ssh.mdx b/cli/reference/attach-ssh.mdx new file mode 100644 index 0000000..7bef9f7 --- /dev/null +++ b/cli/reference/attach-ssh.mdx @@ -0,0 +1,45 @@ +--- +title: "vastai attach ssh" +sidebarTitle: "attach ssh" +--- + +Attach an ssh key to an instance. This will allow you to connect to the instance with the ssh key + +## Usage + +```bash +vastai attach ssh instance_id ssh_key +``` + +## Arguments + + + id of instance to attach to + + + + ssh key to attach to instance + + +## Description + +Attach an ssh key to an instance. This will allow you to connect to the instance with the ssh key. + +## Examples + +```bash +vastai attach "ssh 12371 ssh-rsa AAAAB3NzaC1yc2EAAA..." + vastai attach "ssh 12371 ssh-rsa $(cat ~/.ssh/id_rsa)" +``` + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/cancel-copy.mdx b/cli/reference/cancel-copy.mdx new file mode 100644 index 0000000..dcfaed0 --- /dev/null +++ b/cli/reference/cancel-copy.mdx @@ -0,0 +1,42 @@ +--- +title: "vastai cancel copy" +sidebarTitle: "cancel copy" +--- + +Cancel a remote copy in progress, specified by DST id + +## Usage + +```bash +vastai cancel copy DST +``` + +## Arguments + + + instance_id:/path to target of copy operation + + +## Description + +Use this command to cancel any/all current remote copy operations copying to a specific named instance, given by DST. + +## Examples + +```bash +vast cancel copy 12371 + +The first example cancels all copy operations currently copying data into instance 12371 +``` + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/cancel-maint.mdx b/cli/reference/cancel-maint.mdx new file mode 100644 index 0000000..857ea9a --- /dev/null +++ b/cli/reference/cancel-maint.mdx @@ -0,0 +1,38 @@ +--- +title: "vastai cancel maint" +sidebarTitle: "cancel maint" +description: "Host command" +--- + +Cancel maint window + +This is a **host** command, used for managing machines you are renting out on Vast.ai. + +## Usage + +```bash +vastai cancel maint id +``` + +## Arguments + + + id of machine to cancel maintenance(s) for + + +## Description + +For deleting a machine's scheduled maintenance window(s), use this cancel maint command. +Example: vastai cancel maint 8207 + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/cancel-sync.mdx b/cli/reference/cancel-sync.mdx new file mode 100644 index 0000000..9282352 --- /dev/null +++ b/cli/reference/cancel-sync.mdx @@ -0,0 +1,42 @@ +--- +title: "vastai cancel sync" +sidebarTitle: "cancel sync" +--- + +Cancel a remote copy in progress, specified by DST id + +## Usage + +```bash +vastai cancel sync DST +``` + +## Arguments + + + instance_id:/path to target of sync operation + + +## Description + +Use this command to cancel any/all current remote cloud sync operations copying to a specific named instance, given by DST. + +## Examples + +```bash +vast cancel sync 12371 + +The first example cancels all copy operations currently copying data into instance 12371 +``` + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/change-bid.mdx b/cli/reference/change-bid.mdx new file mode 100644 index 0000000..c2b5ad8 --- /dev/null +++ b/cli/reference/change-bid.mdx @@ -0,0 +1,61 @@ +--- +title: "vastai change bid" +sidebarTitle: "change bid" +--- + +Change the bid price for a spot/interruptible instance + +## Usage + +```bash +vastai change bid id [--price PRICE] +``` + +## Arguments + + + id of instance type to change bid + + +## Options + + + per machine bid price in $/hour + + + + try to schedule a command to run hourly, daily, or monthly. Valid values are HOURLY, DAILY, WEEKLY For ex. --schedule DAILY Choices: `HOURLY`, `DAILY`, `WEEKLY` + + + + Start date/time in format 'YYYY-MM-DD HH:MM:SS PM' (UTC). Default is now. (optional) + + + + End date/time in format 'YYYY-MM-DD HH:MM:SS PM' (UTC). Default is 7 days from now. (optional) + + + + Day of week you want scheduled job to run on (0-6, where 0=Sunday) or "*". Default will be 0. For ex. --day 0 + + + + Hour of day you want scheduled job to run on (0-23) or "*" (UTC). Default will be 0. For ex. --hour 16 + + +## Description + +Change the current bid price of instance id to PRICE. +If PRICE is not specified, then a winning bid price is used as the default. + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/cleanup-machine.mdx b/cli/reference/cleanup-machine.mdx new file mode 100644 index 0000000..3ecf026 --- /dev/null +++ b/cli/reference/cleanup-machine.mdx @@ -0,0 +1,40 @@ +--- +title: "vastai cleanup machine" +sidebarTitle: "cleanup machine" +description: "Host command" +--- + +Remove all expired storage instances from the machine, freeing up space + +This is a **host** command, used for managing machines you are renting out on Vast.ai. + +## Usage + +```bash +vastai cleanup machine ID [options] +``` + +## Arguments + + + id of machine to cleanup + + +## Description + +Instances expire on their end date. Expired instances still pay storage fees, but can not start. +Since hosts are still paid storage fees for expired instances, we do not auto delete them. +Instead you can use this CLI/API function to delete all expired storage instances for a machine. +This is useful if you are running low on storage, want to do maintenance, or are subsidizing storage, etc. + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/clone-volume.mdx b/cli/reference/clone-volume.mdx new file mode 100644 index 0000000..bdca399 --- /dev/null +++ b/cli/reference/clone-volume.mdx @@ -0,0 +1,49 @@ +--- +title: "vastai clone volume" +sidebarTitle: "clone volume" +--- + +Clone an existing volume + +## Usage + +```bash +vastai copy volume [options] +``` + +## Arguments + + + id of volume contract being cloned + + + + id of volume offer volume is being copied to + + +## Options + + + Size of new volume contract, in GB. Must be greater than or equal to the source volume, and less than or equal to the destination offer. (alias: `--size`) + + + + Do not compress volume data before copying. (alias: `--disable_compression`) + + +## Description + +Create a new volume with the given offer, by copying the existing volume. +Size defaults to the size of the existing volume, but can be increased if there is available space. + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/cloud-copy.mdx b/cli/reference/cloud-copy.mdx new file mode 100644 index 0000000..8dfbebb --- /dev/null +++ b/cli/reference/cloud-copy.mdx @@ -0,0 +1,106 @@ +--- +title: "vastai cloud copy" +sidebarTitle: "cloud copy" +--- + +Copy files/folders to and from cloud providers + +## Usage + +```bash +vastai cloud copy --src SRC --dst DST --instance INSTANCE_ID -connection CONNECTION_ID --transfer TRANSFER_TYPE +``` + +## Options + + + path to source of object to copy + + + + path to target of copy operation + + + + id of the instance + + + + id of cloud connection on your account (get from calling 'vastai show connections') + + + + type of transfer, possible options include Instance To Cloud and Cloud To Instance + + + + show what would have been transferred + + + + skip based on size only, not mod-time or checksum + + + + skip all files that exist on destination + + + + skip files that are newer on the destination + + + + delete files on dest excluded from transfer + + + + try to schedule a command to run hourly, daily, or monthly. Valid values are HOURLY, DAILY, WEEKLY For ex. --schedule DAILY Choices: `HOURLY`, `DAILY`, `WEEKLY` + + + + Start date/time in format 'YYYY-MM-DD HH:MM:SS PM' (UTC). Default is now. (optional) + + + + End date/time in format 'YYYY-MM-DD HH:MM:SS PM' (UTC). Default is contract's end. (optional) + + + + Day of week you want scheduled job to run on (0-6, where 0=Sunday) or "*". Default will be 0. For ex. --day 0 + + + + Hour of day you want scheduled job to run on (0-23) or "*" (UTC). Default will be 0. For ex. --hour 16 + + +## Description + +Copies a directory from a source location to a target location. Each of source and destination +directories can be either local or remote, subject to appropriate read and write +permissions required to carry out the action. The format for both src and dst is [instance_id:]path. +You can find more information about the cloud copy operation here: https://vast.ai/docs/gpu-instances/cloud-sync + +## Examples + +```bash +vastai show connections + ID NAME Cloud Type + 1001 test_dir drive + 1003 data_dir drive + + vastai cloud copy --src /folder --dst /workspace --instance 6003036 --connection 1001 --transfer "Instance To Cloud" + +The example copies all contents of /folder into /workspace on instance 6003036 from gdrive connection 'test_dir'. +``` + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/copy.mdx b/cli/reference/copy.mdx new file mode 100644 index 0000000..0b831e8 --- /dev/null +++ b/cli/reference/copy.mdx @@ -0,0 +1,75 @@ +--- +title: "vastai copy" +sidebarTitle: "copy" +--- + +Copy directories between instances and/or local + +## Usage + +```bash +vastai copy SRC DST +``` + +## Arguments + + + Source location for copy operation (supports multiple formats) + + + + Target location for copy operation (supports multiple formats) + + +## Options + + + Location of ssh private key (alias: `--identity`) + + +## Description + +Copies a directory from a source location to a target location. Each of source and destination +directories can be either local or remote, subject to appropriate read and write +permissions required to carry out the action. + +Supported location formats: +- [instance_id:]path (legacy format, still supported) +- C.instance_id:path (container copy format) +- cloud_service:path (cloud service format) +- cloud_service.cloud_service_id:path (cloud service with ID) +- local:path (explicit local path) +- V.volume_id:path (volume copy, see restrictions) + +You should not copy to /root or / as a destination directory, as this can mess up the permissions on your instance ssh folder, breaking future copy operations (as they use ssh authentication) +You can see more information about constraints here: https://vast.ai/docs/gpu-instances/data-movement#constraints +Volume copy is currently only supported for copying to other volumes or instances, not cloud services or local. + +## Examples + +```bash +vast copy 6003036:/workspace/ 6003038:/workspace/ + vast copy C.11824:/data/test local:data/test + vast copy local:data/test C.11824:/data/test + vast copy drive:/folder/file.txt C.6003036:/workspace/ + vast copy s3.101:/data/ C.6003036:/workspace/ + vast copy V.1234:/file C.5678:/workspace/ + +The first example copy syncs all files from the absolute directory '/workspace' on instance 6003036 to the directory '/workspace' on instance 6003038. +The second example copy syncs files from container 11824 to the local machine using structured syntax. +The third example copy syncs files from local to container 11824 using structured syntax. +The fourth example copy syncs files from Google Drive to an instance. +The fifth example copy syncs files from S3 bucket with id 101 to an instance. +``` + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-api-key.mdx b/cli/reference/create-api-key.mdx new file mode 100644 index 0000000..3fee55c --- /dev/null +++ b/cli/reference/create-api-key.mdx @@ -0,0 +1,43 @@ +--- +title: "vastai create api-key" +sidebarTitle: "create api-key" +--- + +Create a new api-key with restricted permissions. Can be sent to other users and teammates + +## Usage + +```bash +vastai create api-key --name NAME --permission_file PERMISSIONS +``` + +## Options + + + name of the api-key + + + + file path for json encoded permissions, see https://vast.ai/docs/cli/roles-and-permissions for more information + + + + optional wildcard key params for advanced keys + + +## Description + +In order to create api keys you must understand how permissions must be sent via json format. +You can find more information about permissions here: https://vast.ai/docs/cli/roles-and-permissions + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-cluster.mdx b/cli/reference/create-cluster.mdx new file mode 100644 index 0000000..ee1c36e --- /dev/null +++ b/cli/reference/create-cluster.mdx @@ -0,0 +1,38 @@ +--- +title: "vastai create cluster" +sidebarTitle: "create cluster" +--- + +Create Vast cluster + +## Usage + +```bash +vastai create cluster SUBNET MANAGER_ID +``` + +## Arguments + + + local subnet for cluster, ex: '0.0.0.0/24' + + + + Machine ID of manager node in cluster. Must exist already. + + +## Description + +Create Vast Cluster by defining a local subnet and manager id. + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-endpoint.mdx b/cli/reference/create-endpoint.mdx new file mode 100644 index 0000000..0089bb5 --- /dev/null +++ b/cli/reference/create-endpoint.mdx @@ -0,0 +1,60 @@ +--- +title: "vastai create endpoint" +sidebarTitle: "create endpoint" +--- + +Create a new endpoint group + +## Usage + +```bash +vastai create endpoint [OPTIONS] +``` + +## Options + + + minimum floor load in perf units/s (token/s for LLms) + + + + minimum floor load in perf units/s (token/s for LLms), but allow handling with cold workers + + + + target capacity utilization (fraction, max 1.0, default 0.9) + + + + cold/stopped instance capacity target as multiple of hot capacity target (default 2.5) + + + + min number of workers to keep 'cold' when you have no load (default 5) + + + + max number of workers your endpoint group can have (default 20) + + + + deployment endpoint name (allows multiple autoscale groups to share same deployment endpoint) + + +## Description + +Create a new endpoint group to manage many autoscaling groups + +Example: vastai create endpoint --target_util 0.9 --cold_mult 2.0 --endpoint_name "LLama" + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-env-var.mdx b/cli/reference/create-env-var.mdx new file mode 100644 index 0000000..be13ffd --- /dev/null +++ b/cli/reference/create-env-var.mdx @@ -0,0 +1,34 @@ +--- +title: "vastai create env-var" +sidebarTitle: "create env-var" +--- + +Create a new user environment variable + +## Usage + +```bash +vastai create env-var +``` + +## Arguments + + + Environment variable name + + + + Environment variable value + + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-instance.mdx b/cli/reference/create-instance.mdx new file mode 100644 index 0000000..fecd769 --- /dev/null +++ b/cli/reference/create-instance.mdx @@ -0,0 +1,170 @@ +--- +title: "vastai create instance" +sidebarTitle: "create instance" +--- + +Create a new instance + +## Usage + +```bash +vastai create instance ID [OPTIONS] [--args ...] +``` + +## Arguments + + + id of instance type to launch (returned from search offers) + + +## Options + + + Create instance from template info + + + + User to use with docker create. This breaks some images, so only use this if you are certain you need it. + + + + size of local disk partition in GB + + + + docker container image to launch + + + + docker login arguments for private repo authentication, surround with '' + + + + label to set on the instance + + + + filename to use as onstart script + + + + contents of onstart script as single argument + + + + override entrypoint for args launch instance + + + + Launch as an ssh instance type + + + + Launch as a jupyter instance instead of an ssh instance + + + + Use (faster) direct connections for jupyter & ssh + + + + For runtype 'jupyter', directory in instance to use to launch jupyter. Defaults to image's working directory + + + + For runtype 'jupyter', Launch instance with jupyter lab + + + + Workaround for images with locale problems: install and generate locales before instance launch, and set locale to C.UTF-8 + + + + Workaround for images with locale problems: set python's locale to C.UTF-8 + + + + env variables and port mapping options, surround with '' + + + + list of arguments passed to container ENTRYPOINT. Onstart is recommended for this purpose. (must be last argument) + + + + Skip sanity checks when creating from an existing instance + + + + Return error if scheduling fails (rather than creating a stopped instance) + + + + (OPTIONAL) create an INTERRUPTIBLE instance with per machine bid price in $/hour + + + + Create a new local volume using an ID returned from the "search volumes" command and link it to the new instance + + + + ID of an existing rented volume to link to the instance during creation. (returned from "show volumes" cmd) + + + + Size of the volume to create in GB. Only usable with --create-volume (default 15GB) + + + + The path to the volume from within the new instance container. e.g. /root/volume + + + + (optional) A name to give the new volume. Only usable with --create-volume + + +## Description + +Performs the same action as pressing the "RENT" button on the website at https://console.vast.ai/create/ +Creates an instance from an offer ID (which is returned from "search offers"). Each offer ID can only be used to create one instance. +Besides the offer ID, you must pass in an '--image' argument as a minimum. + +If you use args/entrypoint launch mode, we create a container from your image as is, without attempting to inject ssh and or jupyter. +If you use the args launch mode, you can override the entrypoint with --entrypoint, and pass arguments to the entrypoint with --args. +If you use --args, that must be the last argument, as any following tokens are consumed into the args string. +For ssh/jupyter launch types, use --onstart-cmd to pass in startup script, instead of --entrypoint and --args. + +## Examples + +```bash +# create an on-demand instance with the PyTorch (cuDNN Devel) template and 64GB of disk +vastai create instance 384826 --template_hash 661d064bbda1f2a133816b6d55da07c3 --disk 64 + +# create an on-demand instance with the pytorch/pytorch image, 40GB of disk, open 8081 udp, direct ssh, set hostname to billybob, and a small onstart script +vastai create instance 6995713 --image pytorch/pytorch --disk 40 --env '-p 8081:8081/udp -h billybob' --ssh --direct --onstart-cmd "env | grep _ >> /etc/environment; echo 'starting up'"; + +# create an on-demand instance with the bobsrepo/pytorch:latest image, 20GB of disk, open 22, 8080, jupyter ssh, and set some env variables +vastai create instance 384827 --image bobsrepo/pytorch:latest --login '-u bob -p 9d8df!fd89ufZ docker.io' --jupyter --direct --env '-e TZ=PDT -e XNAME=XX4 -p 22:22 -p 8080:8080' --disk 20 + +# create an on-demand instance with the pytorch/pytorch image, 40GB of disk, override the entrypoint to bash and pass bash a simple command to keep the instance running. (args launch without ssh/jupyter) +vastai create instance 5801802 --image pytorch/pytorch --disk 40 --onstart-cmd 'bash' --args -c 'echo hello; sleep infinity;' + +# create an interruptible (spot) instance with the PyTorch (cuDNN Devel) template, 64GB of disk, and a bid price of $0.10/hr +vastai create instance 384826 --template_hash 661d064bbda1f2a133816b6d55da07c3 --disk 64 --bid_price 0.1 + +Return value: +Returns a json reporting the instance ID of the newly created instance: +{'success': True, 'new_contract': 7835610} +``` + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-network-volume.mdx b/cli/reference/create-network-volume.mdx new file mode 100644 index 0000000..4f2a555 --- /dev/null +++ b/cli/reference/create-network-volume.mdx @@ -0,0 +1,45 @@ +--- +title: "vastai create network-volume" +sidebarTitle: "create network-volume" +--- + +Create a new network volume + +## Usage + +```bash +vastai create network volume ID [options] +``` + +## Arguments + + + id of network volume offer + + +## Options + + + size in GB of network volume. Default %(default)s GB. (alias: `--size`) + + + + Optional name of network volume. (alias: `--name`) + + +## Description + +Creates a network volume from an offer ID (which is returned from "search network volumes"). Each offer ID can be used to create multiple volumes, +provided the size of all volumes does not exceed the size of the offer. + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-overlay.mdx b/cli/reference/create-overlay.mdx new file mode 100644 index 0000000..affed87 --- /dev/null +++ b/cli/reference/create-overlay.mdx @@ -0,0 +1,38 @@ +--- +title: "vastai create overlay" +sidebarTitle: "create overlay" +--- + +Creates overlay network on top of a physical cluster + +## Usage + +```bash +vastai create overlay CLUSTER_ID OVERLAY_NAME +``` + +## Arguments + + + ID of cluster to create overlay on top of + + + + overlay network name + + +## Description + +Creates an overlay network to allow local networking between instances on a physical cluster + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-ssh-key.mdx b/cli/reference/create-ssh-key.mdx new file mode 100644 index 0000000..af964ff --- /dev/null +++ b/cli/reference/create-ssh-key.mdx @@ -0,0 +1,53 @@ +--- +title: "vastai create ssh-key" +sidebarTitle: "create ssh-key" +--- + +Create a new ssh-key + +## Usage + +```bash +vastai create ssh-key [ssh_public_key] [-y] +``` + +## Arguments + + + add your existing ssh public key to your account (from the .pub file). If no public key is provided, a new key pair will be generated. + + +## Options + + + automatically answer yes to prompts (alias: `--yes`) + + +## Description + +You may use this command to add an existing public key, or create a new ssh key pair and add that public key, to your Vast account. + +If you provide an ssh_public_key.pub argument, that public key will be added to your Vast account. All ssh public keys should be in OpenSSH format. + + Example: $vastai create ssh-key 'ssh_public_key.pub' + +If you don't provide an ssh_public_key.pub argument, a new Ed25519 key pair will be generated. + + Example: $vastai create ssh-key + +The generated keys are saved as ~/.ssh/id_ed25519 (private) and ~/.ssh/id_ed25519.pub (public). Any existing id_ed25519 keys are backed up as .backup_<timestamp>. +The public key will be added to your Vast account. + +All ssh public keys are stored in your Vast account and can be used to connect to instances they've been added to. + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-subaccount.mdx b/cli/reference/create-subaccount.mdx new file mode 100644 index 0000000..1953654 --- /dev/null +++ b/cli/reference/create-subaccount.mdx @@ -0,0 +1,50 @@ +--- +title: "vastai create subaccount" +sidebarTitle: "create subaccount" +--- + +Create a subaccount + +## Usage + +```bash +vastai create subaccount --email EMAIL --username USERNAME --password PASSWORD --type TYPE +``` + +## Options + + + email address to use for login + + + + username to use for login + + + + password to use for login + + + + host/client + + +## Description + +Creates a new account that is considered a child of your current account as defined via the API key. + +vastai create subaccount --email bob@gmail.com --username bob --password password --type host + +vastai create subaccount --email vast@gmail.com --username vast --password password --type host + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-team-role.mdx b/cli/reference/create-team-role.mdx new file mode 100644 index 0000000..6e3c72c --- /dev/null +++ b/cli/reference/create-team-role.mdx @@ -0,0 +1,39 @@ +--- +title: "vastai create team-role" +sidebarTitle: "create team-role" +--- + +Add a new role to your team + +## Usage + +```bash +vastai create team-role --name NAME --permissions PERMISSIONS +``` + +## Options + + + name of the role + + + + file path for json encoded permissions, look in the docs for more information + + +## Description + +Creating a new team role involves understanding how permissions must be sent via json format. +You can find more information about permissions here: https://vast.ai/docs/cli/roles-and-permissions + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-team.mdx b/cli/reference/create-team.mdx new file mode 100644 index 0000000..2a6262c --- /dev/null +++ b/cli/reference/create-team.mdx @@ -0,0 +1,54 @@ +--- +title: "vastai create team" +sidebarTitle: "create team" +--- + +Create a new team + +## Usage + +```bash +vastai create-team --team_name TEAM_NAME +``` + +## Options + + + name of the team + + +## Description + +Creates a new team under your account. + +Unlike legacy teams, this command does NOT convert your personal account into a team. +Each team is created as a separate account, and you can be a member of multiple teams. + +When you create a team: + - You become the team owner. + - The team starts as an independent account with its own billing, credits, and resources. + - Default roles (owner, manager, member) are automatically created. + - You can invite others, assign roles, and manage resources within the team. + +Optional: + You can transfer a portion of your existing personal credits to the team by using + the `--transfer_credit` flag. Example: + vastai create-team --team_name myteam --transfer_credit 25 + +Notes: + - You cannot create a team from within another team account. + +For more details, see: +https://vast.ai/docs/teams-quickstart + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-template.mdx b/cli/reference/create-template.mdx new file mode 100644 index 0000000..bde973b --- /dev/null +++ b/cli/reference/create-template.mdx @@ -0,0 +1,120 @@ +--- +title: "vastai create template" +sidebarTitle: "create template" +--- + +Create a new template + +## Usage + +```bash +vastai create template +``` + +## Options + + + name of the template + + + + docker container image to launch + + + + docker image tag (can also be appended to end of image_path) + + + + link you want to provide + + + + link to repository + + + + docker login arguments for private repo authentication, surround with '' + + + + Contents of the 'Docker options' field + + + + Launch as an ssh instance type + + + + Launch as a jupyter instance instead of an ssh instance + + + + Use (faster) direct connections for jupyter & ssh + + + + For runtype 'jupyter', directory in instance to use to launch jupyter. Defaults to image's working directory + + + + For runtype 'jupyter', Launch instance with jupyter lab + + + + contents of onstart script as single argument + + + + search offers filters + + + + Disable default search param query args (alias: `--no-default`) + + + + disk storage space, in GB + + + + readme string + + + + hide the readme from users + + + + description string + + + + make template available to public + + +## Description + +Create a template that can be used to create instances with + +## Examples + +```bash +vastai create template --name "tgi-llama2-7B-quantized" --image "ghcr.io/huggingface/text-generation-inference:1.0.3" + --env "-p 3000:3000 -e MODEL_ARGS='--model-id TheBloke/Llama-2-7B-chat-GPTQ --quantize gptq'" + --onstart-cmd 'wget -O - https://raw.githubusercontent.com/vast-ai/vast-pyworker/main/scripts/launch_tgi.sh | bash' + --search_params "gpu_ram>=23 num_gpus=1 gpu_name=RTX_3090 inet_down>128 direct_port_count>3 disk_space>=192 driver_version>=535086005 rented=False" + --disk_space 8.0 --ssh --direct +``` + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-volume.mdx b/cli/reference/create-volume.mdx new file mode 100644 index 0000000..d9e070f --- /dev/null +++ b/cli/reference/create-volume.mdx @@ -0,0 +1,45 @@ +--- +title: "vastai create volume" +sidebarTitle: "create volume" +--- + +Create a new volume + +## Usage + +```bash +vastai create volume ID [options] +``` + +## Arguments + + + id of volume offer + + +## Options + + + size in GB of volume. Default %(default)s GB. (alias: `--size`) + + + + Optional name of volume. (alias: `--name`) + + +## Description + +Creates a volume from an offer ID (which is returned from "search volumes"). Each offer ID can be used to create multiple volumes, +provided the size of all volumes does not exceed the size of the offer. + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/create-workergroup.mdx b/cli/reference/create-workergroup.mdx new file mode 100644 index 0000000..28cf85f --- /dev/null +++ b/cli/reference/create-workergroup.mdx @@ -0,0 +1,84 @@ +--- +title: "vastai create workergroup" +sidebarTitle: "create workergroup" +--- + +Create a new autoscale group + +## Usage + +```bash +vastai workergroup create [OPTIONS] +``` + +## Options + + + template hash (required, but **Note**: if you use this field, you can skip search_params, as they are automatically inferred from the template) + + + + template id (optional) + + + + Disable default search param query args (alias: `--no-default`) + + + + launch args string for create instance ex: "--onstart onstart_wget.sh --env '-e ONSTART_PATH=https://s3.amazonaws.com/vast.ai/onstart_OOBA.sh' --image atinoda/text-generation-webui:default-nightly --disk 64" + + + + deployment endpoint name (allows multiple workergroups to share same deployment endpoint) + + + + deployment endpoint id (allows multiple workergroups to share same deployment endpoint) + + + + number of workers to create to get an performance estimate for while initializing workergroup (default 3) + + + + estimated GPU RAM req (independent of search string) + + + + search param string for search offers ex: "gpu_ram>=23 num_gpus=2 gpu_name=RTX_4090 inet_down>200 direct_port_count>2 disk_space>=64" + + + + [NOTE: this field isn't currently used at the workergroup level] minimum floor load in perf units/s (token/s for LLms) + + + + [NOTE: this field isn't currently used at the workergroup level] target capacity utilization (fraction, max 1.0, default 0.9) + + + + [NOTE: this field isn't currently used at the workergroup level]cold/stopped instance capacity target as multiple of hot capacity target (default 2.0) + + + + min number of workers to keep 'cold' for this workergroup + + +## Description + +Create a new autoscaling group to manage a pool of worker instances. + +Example: vastai create workergroup --template_hash HASH --endpoint_name "LLama" --test_workers 5 + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/defrag-machines.mdx b/cli/reference/defrag-machines.mdx new file mode 100644 index 0000000..0e9cee5 --- /dev/null +++ b/cli/reference/defrag-machines.mdx @@ -0,0 +1,37 @@ +--- +title: "vastai defrag machines" +sidebarTitle: "defrag machines" +description: "Host command" +--- + +Defragment machines + +This is a **host** command, used for managing machines you are renting out on Vast.ai. + +## Usage + +```bash +vastai defragment machines IDs +``` + +## Arguments + + + ids of machines + + +## Description + +Defragment some of your machines. This will rearrange GPU assignments to try and make more multi-gpu offers available. + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/delete-api-key.mdx b/cli/reference/delete-api-key.mdx new file mode 100644 index 0000000..7cb72a4 --- /dev/null +++ b/cli/reference/delete-api-key.mdx @@ -0,0 +1,30 @@ +--- +title: "vastai delete api-key" +sidebarTitle: "delete api-key" +--- + +Remove an api-key + +## Usage + +```bash +vastai delete api-key ID +``` + +## Arguments + + + id of apikey to remove + + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/delete-cluster.mdx b/cli/reference/delete-cluster.mdx new file mode 100644 index 0000000..dde78a3 --- /dev/null +++ b/cli/reference/delete-cluster.mdx @@ -0,0 +1,34 @@ +--- +title: "vastai delete cluster" +sidebarTitle: "delete cluster" +--- + +Delete Cluster + +## Usage + +```bash +vastai delete cluster CLUSTER_ID +``` + +## Arguments + + + ID of cluster to delete + + +## Description + +Delete Vast Cluster + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/delete-endpoint.mdx b/cli/reference/delete-endpoint.mdx new file mode 100644 index 0000000..0b61dcc --- /dev/null +++ b/cli/reference/delete-endpoint.mdx @@ -0,0 +1,34 @@ +--- +title: "vastai delete endpoint" +sidebarTitle: "delete endpoint" +--- + +Delete an endpoint group + +## Usage + +```bash +vastai delete endpoint ID +``` + +## Arguments + + + id of endpoint group to delete + + +## Description + +Example: vastai delete endpoint 4242 + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/delete-env-var.mdx b/cli/reference/delete-env-var.mdx new file mode 100644 index 0000000..ba0b45c --- /dev/null +++ b/cli/reference/delete-env-var.mdx @@ -0,0 +1,30 @@ +--- +title: "vastai delete env-var" +sidebarTitle: "delete env-var" +--- + +Delete a user environment variable + +## Usage + +```bash +vastai delete env-var +``` + +## Arguments + + + Environment variable name to delete + + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/delete-machine.mdx b/cli/reference/delete-machine.mdx new file mode 100644 index 0000000..476cccb --- /dev/null +++ b/cli/reference/delete-machine.mdx @@ -0,0 +1,33 @@ +--- +title: "vastai delete machine" +sidebarTitle: "delete machine" +description: "Host command" +--- + +Delete machine if the machine is not being used by clients. host jobs on their own machines are disregarded and machine is force deleted. + +This is a **host** command, used for managing machines you are renting out on Vast.ai. + +## Usage + +```bash +vastai delete machine +``` + +## Arguments + + + id of machine to delete + + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/delete-overlay.mdx b/cli/reference/delete-overlay.mdx new file mode 100644 index 0000000..0610700 --- /dev/null +++ b/cli/reference/delete-overlay.mdx @@ -0,0 +1,30 @@ +--- +title: "vastai delete overlay" +sidebarTitle: "delete overlay" +--- + +Deletes overlay and removes all of its associated instances + +## Usage + +```bash +vastai delete overlay OVERLAY_IDENTIFIER +``` + +## Arguments + + + ID (int) or name (str) of overlay to delete + + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/delete-scheduled-job.mdx b/cli/reference/delete-scheduled-job.mdx new file mode 100644 index 0000000..b02764a --- /dev/null +++ b/cli/reference/delete-scheduled-job.mdx @@ -0,0 +1,30 @@ +--- +title: "vastai delete scheduled-job" +sidebarTitle: "delete scheduled-job" +--- + +Delete a scheduled job + +## Usage + +```bash +vastai delete scheduled-job ID +``` + +## Arguments + + + id of scheduled job to remove + + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/delete-ssh-key.mdx b/cli/reference/delete-ssh-key.mdx new file mode 100644 index 0000000..b24e814 --- /dev/null +++ b/cli/reference/delete-ssh-key.mdx @@ -0,0 +1,30 @@ +--- +title: "vastai delete ssh-key" +sidebarTitle: "delete ssh-key" +--- + +Remove an ssh-key + +## Usage + +```bash +vastai delete ssh-key ID +``` + +## Arguments + + + id ssh key to delete + + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/delete-template.mdx b/cli/reference/delete-template.mdx new file mode 100644 index 0000000..2b62f16 --- /dev/null +++ b/cli/reference/delete-template.mdx @@ -0,0 +1,40 @@ +--- +title: "vastai delete template" +sidebarTitle: "delete template" +--- + +Delete a Template + +## Usage + +```bash +vastai delete template [--template-id | --hash-id ] +``` + +## Options + + + Template ID of Template to Delete + + + + Hash ID of Template to Delete + + +## Description + +Note: Deleting a template only removes the user's replationship to a template. It does not get destroyed +Example: vastai delete template --template-id 12345 +Example: vastai delete template --hash-id 49c538d097ad6437413b83711c9f61e8 + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/delete-volume.mdx b/cli/reference/delete-volume.mdx new file mode 100644 index 0000000..101874b --- /dev/null +++ b/cli/reference/delete-volume.mdx @@ -0,0 +1,34 @@ +--- +title: "vastai delete volume" +sidebarTitle: "delete volume" +--- + +Delete a volume + +## Usage + +```bash +vastai delete volume ID +``` + +## Arguments + + + id of volume contract + + +## Description + +Deletes volume with the given ID. All instances using the volume must be destroyed before the volume can be deleted. + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/delete-workergroup.mdx b/cli/reference/delete-workergroup.mdx new file mode 100644 index 0000000..6992785 --- /dev/null +++ b/cli/reference/delete-workergroup.mdx @@ -0,0 +1,35 @@ +--- +title: "vastai delete workergroup" +sidebarTitle: "delete workergroup" +--- + +Delete a workergroup group + +## Usage + +```bash +vastai delete workergroup ID +``` + +## Arguments + + + id of group to delete + + +## Description + +Note that deleting a workergroup doesn't automatically destroy all the instances that are associated with your workergroup. +Example: vastai delete workergroup 4242 + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/destroy-instance.mdx b/cli/reference/destroy-instance.mdx new file mode 100644 index 0000000..ed4b76e --- /dev/null +++ b/cli/reference/destroy-instance.mdx @@ -0,0 +1,35 @@ +--- +title: "vastai destroy instance" +sidebarTitle: "destroy instance" +--- + +Destroy an instance (irreversible, deletes data) + +## Usage + +```bash +vastai destroy instance id [-h] [--api-key API_KEY] [--raw] +``` + +## Arguments + + + id of instance to delete + + +## Description + +Perfoms the same action as pressing the "DESTROY" button on the website at https://console.vast.ai/instances/ +Example: vastai destroy instance 4242 + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/destroy-instances.mdx b/cli/reference/destroy-instances.mdx new file mode 100644 index 0000000..3ce369c --- /dev/null +++ b/cli/reference/destroy-instances.mdx @@ -0,0 +1,30 @@ +--- +title: "vastai destroy instances" +sidebarTitle: "destroy instances" +--- + +Destroy a list of instances (irreversible, deletes data) + +## Usage + +```bash +vastai destroy instances [--raw] +``` + +## Arguments + + + ids of instance to destroy + + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/destroy-team.mdx b/cli/reference/destroy-team.mdx new file mode 100644 index 0000000..8bd868f --- /dev/null +++ b/cli/reference/destroy-team.mdx @@ -0,0 +1,24 @@ +--- +title: "vastai destroy team" +sidebarTitle: "destroy team" +--- + +Destroy your team + +## Usage + +```bash +vastai destroy team +``` + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/detach-ssh.mdx b/cli/reference/detach-ssh.mdx new file mode 100644 index 0000000..055f49b --- /dev/null +++ b/cli/reference/detach-ssh.mdx @@ -0,0 +1,38 @@ +--- +title: "vastai detach ssh" +sidebarTitle: "detach ssh" +--- + +Detach an ssh key from an instance + +## Usage + +```bash +vastai detach instance_id ssh_key_id +``` + +## Arguments + + + id of the instance + + + + id of the key to detach to the instance + + +## Description + +Example: vastai detach 99999 12345 + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/execute.mdx b/cli/reference/execute.mdx new file mode 100644 index 0000000..555bc5b --- /dev/null +++ b/cli/reference/execute.mdx @@ -0,0 +1,71 @@ +--- +title: "vastai execute" +sidebarTitle: "execute" +--- + +Execute a (constrained) remote command on a machine + +## Usage + +```bash +vastai execute id COMMAND +``` + +## Arguments + + + id of instance to execute on + + + + bash command surrounded by single quotes + + +## Options + + + try to schedule a command to run hourly, daily, or monthly. Valid values are HOURLY, DAILY, WEEKLY For ex. --schedule DAILY Choices: `HOURLY`, `DAILY`, `WEEKLY` + + + + Start date/time in format 'YYYY-MM-DD HH:MM:SS PM' (UTC). Default is now. (optional) + + + + End date/time in format 'YYYY-MM-DD HH:MM:SS PM' (UTC). Default is 7 days from now. (optional) + + + + Day of week you want scheduled job to run on (0-6, where 0=Sunday) or "*". Default will be 0. For ex. --day 0 + + + + Hour of day you want scheduled job to run on (0-23) or "*" (UTC). Default will be 0. For ex. --hour 16 + + +## Description + +Examples: + vastai execute 99999 'ls -l -o -r' + vastai execute 99999 'rm -r home/delete_this.txt' + vastai execute 99999 'du -d2 -h' + +available commands: + ls List directory contents + rm Remote files or directories + du Summarize device usage for a set of files + +Return value: +Returns the output of the command which was executed on the instance, if successful. May take a few seconds to retrieve the results. + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/get-endpt-logs.mdx b/cli/reference/get-endpt-logs.mdx new file mode 100644 index 0000000..4a19751 --- /dev/null +++ b/cli/reference/get-endpt-logs.mdx @@ -0,0 +1,44 @@ +--- +title: "vastai get endpt-logs" +sidebarTitle: "get endpt-logs" +--- + +Fetch logs for a specific serverless endpoint group + +## Usage + +```bash +vastai get endpt-logs ID [--api-key API_KEY] +``` + +## Arguments + + + id of endpoint group to fetch logs from + + +## Options + + + log detail level (0 to 3) + + + + + + +## Description + +Example: vastai get endpt-logs 382 + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/get-wrkgrp-logs.mdx b/cli/reference/get-wrkgrp-logs.mdx new file mode 100644 index 0000000..3dc6bb6 --- /dev/null +++ b/cli/reference/get-wrkgrp-logs.mdx @@ -0,0 +1,44 @@ +--- +title: "vastai get wrkgrp-logs" +sidebarTitle: "get wrkgrp-logs" +--- + +Fetch logs for a specific serverless worker group group + +## Usage + +```bash +vastai get wrkgrp-logs ID [--api-key API_KEY] +``` + +## Arguments + + + id of endpoint group to fetch logs from + + +## Options + + + log detail level (0 to 3) + + + + + + +## Description + +Example: vastai get endpt-logs 382 + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/invite-member.mdx b/cli/reference/invite-member.mdx new file mode 100644 index 0000000..1054659 --- /dev/null +++ b/cli/reference/invite-member.mdx @@ -0,0 +1,34 @@ +--- +title: "vastai invite member" +sidebarTitle: "invite member" +--- + +Invite a team member + +## Usage + +```bash +vastai invite member --email EMAIL --role ROLE +``` + +## Options + + + email of user to be invited + + + + role of user to be invited + + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/join-cluster.mdx b/cli/reference/join-cluster.mdx new file mode 100644 index 0000000..cecd51e --- /dev/null +++ b/cli/reference/join-cluster.mdx @@ -0,0 +1,38 @@ +--- +title: "vastai join cluster" +sidebarTitle: "join cluster" +--- + +Join Machine to Cluster + +## Usage + +```bash +vastai join cluster CLUSTER_ID MACHINE_IDS +``` + +## Arguments + + + ID of cluster to add machine to + + + + machine id(s) to join cluster + + +## Description + +Join's Machine to Vast Cluster + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/join-overlay.mdx b/cli/reference/join-overlay.mdx new file mode 100644 index 0000000..5285f9b --- /dev/null +++ b/cli/reference/join-overlay.mdx @@ -0,0 +1,38 @@ +--- +title: "vastai join overlay" +sidebarTitle: "join overlay" +--- + +Adds instance to an overlay network + +## Usage + +```bash +vastai join overlay OVERLAY_NAME INSTANCE_ID +``` + +## Arguments + + + Overlay network name to join instance to. + + + + Instance ID to add to overlay. + + +## Description + +Adds an instance to a compatible overlay network. + +## Global Options + +The following options are available for all commands: + +| Option | Description | +| --- | --- | +| `--url URL` | Server REST API URL | +| `--retry N` | Retry limit | +| `--raw` | Output machine-readable JSON | +| `--explain` | Verbose explanation of API calls | +| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) | diff --git a/cli/reference/label-instance.mdx b/cli/reference/label-instance.mdx new file mode 100644 index 0000000..dd2e096 --- /dev/null +++ b/cli/reference/label-instance.mdx @@ -0,0 +1,34 @@ +--- +title: "vastai label instance" +sidebarTitle: "label instance" +--- + +Assign a string label to an instance + +## Usage + +```bash +vastai label instance