nice Python API

[leshy]

some basic things people and agents should be able to quickly code and run

run a blueprint, and iteract with it via rpc calls to modules
run a skill programatically (I want to trigger/stop people follow)
run a skill from a running blueprint from a py script, get updates from the skill
check if there is an obstacle in front of the robot (and how far)
build a 360 2d occupancy histogram from the robots perspective
turning 90 deg, 180deg etc
move forward until you reach obstacle

some random things just to confirm motion control is on point like "walk around an object facing it, take photos, make a gif" (build a 360 view of something)

person follow, security patrol should be a super easy oneliner type things

robot.move(detector("person", robot.camera).project(robot.lidar).to_vector())

get latest anything - frame, global map etc

if any of the above is not super easy, can implement small fixes until it is,

look at lego mindstorms API
https://lego.github.io/MINDSTORMS-Robot-Inventor-hub-API/mod_motion.html
simple game engines for character control
https://www.love2d.org/

if there is time small useful apps like,

[spomichter]

Will implement first blueprint level API as first pass and then module level API later. Because additional changes to core needed to handle LCM init for example if im just running bla = dimos.connect("UnitreeGo2Connection") unless we just make sure the agent inits LCM via the command line itself?

So proposed initial pass:

Simple Example

import dimos

# Normal start and connect
unitree_app = dimos()
> Returns static PER APP UUID inline for agent to save in memory

# Run from all_blueprints.py
unitree_app.run("unitree-g2-basic-blueprint")

# Run as a blueprint object
unitree_app.run(unitree_blueprint)

# Add a additional module that autoconnects dynamically, Returns failed topics that cant find connections
unitree_app.run(yolo_detector_module)

print(app.get_skills())
unitree_app.run_skill("observe")()
> returns string observation

unitree_app.run_skill("relative_move")(distance=1m)

Add another module to the app from above after the fact

# Connect to existing app from above 

# Agent forgot what the UUID was lets check 
dimos ls
> Returns latest UUID
unitree_app_2 = dimos(uuid)
unitree_app_2.run("openclaw-agent")

Creating a module inline / import new module

# Option 1: write a module inline
class Module1(Module):
 ...

# Option 2: import 
import dimos
import yolo_detection_module

# NEW dimos application
drone_app = dimos()

app.run(yolo_detection_module)

app.remapping(/video, /color_image)

app.stop()

Other utilities (pretend they are python)

dimos ls
> App ID | blueprint name | time_started | time running | active/inactive
> xxxx         | bla1
> xxxx          

dimos.killall # forced
dimos.stopall # graceful 

# App operations
dimos <app-id> list
> Module ID | blueprint name | time_started | time running | active/inactive
> xxxx         | bla1
> xxxx          
dimos <app-id> log tail
dimos <app-id> log head <lines>
dimos <app-id> stop
dimos <app-id> kill
dimos <app-id> restart

# Module operations
dimos <app-id> <module-id> log tail
dimos <app-id> <module-id> log head <lines>
dimos <app-id> <module-id> stop
dimos <app-id> <module-id> kill
dimos <app-id> <module-id> restart

# Remapping 
dimos <app-id> remap.('/video', '/color_image')

[spomichter]

Dimensional Applications

Application (top level — live, long-running instance)
  └── Blueprint (predefined module groups / "run files")
    └── Module (pub/sub primitive)

Key properties of Applications:

1. Composition: An Application can compose multiple blueprints + individual modules. e.g. one blueprint + five standalone modules, or two blueprints together.

2. Long-running: Applications persist and you can hot-add modules to a running application as you debug/iterate.

3. Transport & Remapping: Applications should be able to do transport mapping and remappings, similar to how blueprints do it today. (Remapping at application level is a future pass.)

4. Relationship to Blueprints: Blueprints are templates / "run files" that people define. Applications are live instances that consume blueprints. As you construct applications, they can functionally replace blueprints in many cases, but blueprints remain the portable, shareable unit.

5. Modules as primitives: Modules remain the core pub/sub primitive. Applications orchestrate them.

How this maps to the Python API:

import dimos

# Create an application
app = dimos()

# Run a blueprint (adds all its modules to the app)
app.run("unitree-go2-basic")

# Hot-add a standalone module
app.run(yolo_detector_module)

# Compose multiple blueprints
app.run("unitree-go2-basic")
app.run("openclaw-agent")

# Skills come from all modules in the app
app.get_skills()
app.run_skill("observe")()

app.stop()

[leshy]

commented here

#1720

we can tune this API and agree on it but a few issues:

1 - paul is working on something very close to this now.

can spec out exact API with him and we are good, we were well under way with this @ #1665

(you can write wishlist like "I want to interact with active blueprints from another script" etc)

2 - I wouldn't do this without specing out multi embodiment

let's not rewrite again in a month, let's agree on multi embodiment deploy API (I want 2 go2 modules, talking to a single global mapper, etc - actual core modules support this, blueprints should too)

3 - This is just a deployment API, actual py API task is deeper

above proposes to solve 3 things in my issue

run a blueprint, and iteract with it via rpc calls to modules
run a skill programatically (I want to trigger/stop people follow)
run a skill from a running blueprint from a py script, get updates from the skill

but issue has a bunch of other things (check the top), in terms of actual robot usage, this involves hand writing a bunch of modules, and making sure they are mostly 5-liners that kids can understand

How to proceed

I can take 1 and 2 with Paul, and we can make a nice thing here, confirm with you

you take 3 start thinking about solving simple tasks make sure you can actually use the robot (otherwise agent won't be able to either!) like why isn't person follow a oneliner? how do I sit next to a couch? this is a big isolated work