[REQUEST] OASIS integration for Home Assistant

[Hedda]

Please make an OASIS integration (custom component) for Home Assistant to enable computer vision interaction + input streaming.

• https://developers.home-assistant.io/

  • https://developers.home-assistant.io/docs/development_index/

  • https://developers.home-assistant.io/docs/creating_component_index/

FYI, also posted this for discussion on Home Assistant community forums:

• https://community.home-assistant.io/t/oasis-smart-home-operating-system-based-on-ros-2-robot-operating-system/405110

I believe that most other somewhat similar integrations for Home Assistant use their WebSocket API or REST API (and MQTT) to communicate.

PS: Alternatively, consider extending device types in existing Kodi integration to take input from this and other inputs via it instead:

• https://www.home-assistant.io/integrations/kodi

[eigendude]

I started experimenting with Home Assistant integration so that I can talk to my Philips Hue bulbs, but now I'm waiting for Matter to launch and I'll add Matter support directly.

All I need is another use case that HA would provide. If one pops up then you can expect integration.

[Hedda]

I started experimenting with Home Assistant integration so that I can talk to my Philips Hue bulbs

Tip for directly connecting Zigbee based Philips Hue lights to Home Assistant is to buy a Silicon Labs EFR32MG21 (or EFR32MG13) based USB adapter (like Elelabs/Popp Zigbee USB stick) or a Texas Instruments CC2652P/CC2652R based Zigbee USB adapter (like "Itead Sonoff Zigbee 3.0 USB Dongle Plus") and connect that via a long external USB extension cable (to get it away from any electonics so they do not interfer with signals) and then use Home Assistant's native ZHA integration component -> https://www.home-assistant.io/integrations/zha/

I' m waiting for Matter to launch and I'll add Matter support directly.

I believe that a more popular option for OpenThread Board Router is to get a Silicon Labs EFR32MG21 (alternatively older EFR32MG13 or newer EFR32MG24) based dongle but those are more expensive and harder to find today due to the current chip shortage (which Silabs has suffered most), see example:

Regardless, recommend that you consider buying several of those "Itead Sonoff Zigbee 3.0 USB Dongle Plus" adapters since they are very inexpensive and not only can you use it as a Zigbee Coordinator adapter as well as flash them as Zigbee Router device firmware (to act as a repeater for extending Zigbee range and coverage) but you should also be able to flash them with OpenThread Board Router (OTBR) firmware (to be used as a bridge for Matter over Thread devices).

• https://github.com/NabuCasa/universal-silabs-flasher

Home Assistant does however not yet have a Matter integration for Home Assistant the founders who now employ some deevelopers under Nabu Casa mentioned in the latest newsletter that they have employed the lead developer from the Z-Wave JS project might help get it started -> https://building.open-home.io/a-new-home-for-our-newsletter/

Another Home Assistant developer also wrote that he started working on both an OpenThread Board Router addon and a Silicon Labs multi-protocol addon for Thread and Zigbee which can be used on the same dongle at same time if using an "RPC" firmware:

zigpy/zigpy#894

https://github.com/home-assistant/addons-development/tree/master/silabs-multiprotocol

https://github.com/home-assistant/addons-development/tree/master/openthread_border_router

Previously I believe those using OpenThread Board Router (OTBR) firmware on Silicon Labs EFR32 (EFR32MG21 or EFR32MG13) or CC2652P (and CC2652R or CC2652RB) are only communicate with OpenThread devices simply by using their IPv6 addresses, controlling and publish a local OpenThread network Thread/OpenThread togther with the wpantund Userspace WPAN Network Daemon:

https://github.com/openthread/wpantund

All I need is another use case that HA would provide. If one pops up then you can expect integration.

FYI, I know that many in the Home Assistant userbase may not actually use Home Assistant itself as an automation platform, (as that part of Home Assistant still is neither very user-friendly or extensive out-of-the-box if used via the UI), therefor many in the Home Assistant userbase might instead use Node-RED or third-party scripting for automations and only utilize Home Assistant as an integration platform and protocol bridge/proxy to consolidate all your devices because Home Assistant already has a very extensive collection of integrations for most things.

Home Assistant Analytics statistics even show that the average Home Assistant instance has 76 integrations installed (that of course include several default system integrations as well) -> https://analytics.home-assistant.io/#stats

By the way, be sure to check out HACS (Home Assistant Community Store) for additional integrations -> https://hacs.xyz

[jatgm1]

Hey I need to connect my moorebot scout (ros1melodic) to my smart home to get images (Preferably in color) and send commands to and get data from to use in automatons. Like alert triggered, send scout #1 to door. Or if motion in kitchen use path to kitchen. I really also want to just straight up control it from a home assistant dashboard with the buttons for forward, backwards, and turning. Just in case I want to give selective access to someone. Also because theirs no other way to do it in the app. Also to use it as a motion sensor. This seems like a fantastic idea that would make my smart home far more reactive and likely to drive away intruders with sheer confusion from multiple robots converging on their position.

[eigendude]

Hey I need to connect my moorebot scout (ros1melodic) to my smart home to get images (Preferably in color) and send commands to and get data from to use in automatons.

Sorry, I don't support EOL versions of ROS.

Tip for directly connecting Zigbee based Philips Hue lights to Home Assistant is to buy a Silicon Labs EFR32MG21 (or EFR32MG13) based USB adapter (like Elelabs/Popp Zigbee USB stick) or a Texas Instruments CC2652P/CC2652R based Zigbee USB adapter (like "Itead Sonoff Zigbee 3.0 USB Dongle Plus") and connect that via a long external USB extension cable (to get it away from any electonics so they do not interfer with signals) and then use Home Assistant's native ZHA integration component -> https://www.home-assistant.io/integrations/zha/

I have renewed interest in connecting Hue to OASIS. I'll start looking for solutions again.

[Hedda]

Custom component integration for Home Assistant’s core (backend) would be very cool to see some day, and made available via publishing it on HACS (Home Assistant’s Community Store) to make it so that more people discover it:

• https://developers.home-assistant.io/docs/creating_component_index/

  • https://developers.home-assistant.io/docs/architecture_components/

• https://www.hacs.xyz

  • https://www.hacs.xyz/docs/contribute/

    • https://www.hacs.xyz/docs/publish/integration/

A good template to use as a reference is blueprint. You can generate a template similar to blueprint and customize it to your context by using cookiecutter-homeassistant-custom-component.

[Hedda]

@eigendude Again, regardless of you creating an custom OASIS integration component or not, I highly recommend you look into migrating away from your Philips Hue Bridge and move to the native ZHA (Zigbee Home Automation) integration in Home Assistant:

• https://community.home-assistant.io/t/migrating-from-a-philips-hue-bridge-to-skyconnect/620862

While many Home Assistant users do have Philips Hue Bridge there are a lot more users who are using that native ZHA integration:

• https://analytics.home-assistant.io/integrations/

All need for ZHA (Zigbee Home Automation) is a Zigbee Coordinator USB adapter (Zigbee radio) + a long USB extension cable:

• https://community.home-assistant.io/t/zigbee-buyers-guide/654695

Suggest just buying the official Home Assistant Connect ZBT-1 (a.k.a. SkyConnect) because it is easiest to update and maintain:

• https://www.home-assistant.io/connectzbt1/

Best alternative option today is otherwise Sonoff ZBDongle-E and/or Sonoff ZBDongle-P.

• https://sonoff.tech/product/gateway-and-sensors/sonoff-zigbee-3-0-usb-dongle-plus-e/

• https://sonoff.tech/product/gateway-and-sensors/sonoff-zigbee-3-0-usb-dongle-plus-p/

[eigendude]

I'd like to ditch the Hue Bridge for a more open (and far cheaper!) solution. It might be a good summertime project to switch my lighting infrastructure over to a new system.

My current goal is apartment-wide person detection to control the lights for when I do get them automated. I've scattered a combination of 10-year-old laptops with webcams, gaming cameras, and raspberry pi cameras around the place. I'm piping the video feeds to a central server doing pose detection on all images in parallel using MediaPipe. I have two detection tasks left before I start on lighting:

• Modify the Kinect2 driver to downscale the video
• Add infrared lighting so the NO-IR cameras work in the dark.

Here's a NO-IR raspberry pi camera I embedded in some LEGO infrastructure doing pose detection in realtime:

After detection is ready I'll work on lighting.

[Hedda]

I'd like to ditch the Hue Bridge for a more open (and far cheaper!) solution. It might be a good summertime project to switch my lighting infrastructure over to a new system.

If you want an inexpensive dedicated home automation as an entry-level appliance entry-level that is super easy to get started with and can fully replace your Philips Hue Bridge then I strongly recommend buying the official Home Assistant Green in combination with their official Home Assistant Connect ZBT-1 (formerly "SkyConnect"). The main benefits with this is that it is very simple to maintain since it is a dedicated appliance, and with tha anyone can be up and running with their official built-in native ZHA (Zigbee Home Automation) integration within 30-minutes. That allows you to integrate all Zigbee compatible devices, so not just Philips Hue products, (as well as have the possibility of adding additional USB radio adapters connecting Thread-based Matter devices and Z-Wave devices too if you want later).

However, if what you are looking for a more flexible solution then suggest to run Home Assistant Operating System (Home Assistant OS) in a Virtual Machine (VM) on a more powerful computer/server (which is turn is running Proxmox Virtual Environment (PVE) virtualization hypervisor). If you go with that instead then you can pass-though USB radio adapters to the VM with Home Assistant OS. The main benefits with that is you can run a fully sandboxed operating-system in each virtual machine, (which can also make quick backup and retro easier if you use VM snapshots in combination with traditional backups).

Again, if you want a cheaper Zigbee USB radio adapter that is still great then buy the Sonoff ZBDongle-E and/or Sonoff ZBDongle-P:

• https://itead.cc/product/zigbee-3-0-usb-dongle/
• https://itead.cc/product/sonoff-zigbee-3-0-usb-dongle-plus/

PS: Another tip is that ITead that makes Sonoff devices have loads of cheap Zigbee devices and many other smart home products:

• https://itead.cc/smart-home/

[eigendude]

OK, finished Kinect 2 support enough for pose detection in MediaPipe: https://github.com/eigendude/oasis_kinect2

I integrated HA into OASIS: https://github.com/eigendude/OASIS/tree/main/oasis_ha. I installed it on a spare RPi 4. I set up all the services provided out-of-the-box. It added my Hue devices and rooms.

Again, highly recommend you migrate from a Philips Hue Bridge to the native ZHA (Zigbee Home Automation) integration in Home Assistant

I looked into direct zigbee comms, and went with a Hue Hub, because while expensive it's high quality infrastructure. Being able to store state safely on the network is valuable. ZHA would need to keep state on-device, with all the difficulties that brings.

Home Assistant does however not yet have a Matter integration

I was disappointed when Matter came out. It sounds like it's far more dependent on the internet that I thought it would be.

I strongly recommend buying the official Home Assistant Green in combination with their official Home Assistant Connect ZBT-1 (formerly "SkyConnect").

If I knew about Home Assistant Green before I bought the Hue Bridge I'd definitely have gone with that. But now that I have the bridge, and it's a valuable device, I feel like I should continue using it.

Custom component integration for Home Assistant’s core (backend) would be very cool to see some day, and made available via publishing it on HACS (Home Assistant’s Community Store) to make it so that more people discover it

Now that I have HA setup I need to integrate it with the pose detector. The problem with a HA component is the ROS dependency, whereas a ROS component simply has a HA dependency.

I'll figure out the best way to integrate.

What integration ideas do you have?

[eigendude]

It looks like MQTT is the protocol most suited for IoT. I found a ROS 2 MQTT bridge. I'll see if I can get it to work.

[Hedda]

Believe that WebSocket API connection is usually prefer for real-time bi-direction connections to native Home Assistant integrations:

• https://www.home-assistant.io/integrations/websocket_api/

  • https://developers.home-assistant.io/docs/api/websocket/

Found these video tutorials on WebSocket API in Home Assistant via a quick search:

• https://www.youtube.com/watch?v=Qk9A0QbG5-I

  • https://quickstarts.postman.com/guide/home-assistant/index.html?index=..%2F..index#0

• https://www.youtube.com/watch?v=Q5k2lj2iskw

• https://www.youtube.com/watch?v=34yD15jmeEA

Note! They have an official "thick" JavaScript WebSocket client for Home Assistant that can maybe also be used as a reference(?):

https://github.com/home-assistant/home-assistant-js-websocket

REST (RESTful) is an alternative but understand that API is for more quick and dirty connections instead of a proper integration(?):

• https://www.home-assistant.io/integrations/rest/

  • https://developers.home-assistant.io/docs/api/rest/

Maybe could also check out ESPHome's Native API Component and ESPHome native API? It too has many advantages over MQTT:

• https://esphome.io/components/api.html

  • https://esphome.io/components/api.html#advantages-over-mqtt

Also, while very old and probably outdated today, but FYI, also had this old Home Assistant client for MicroPython as an example:

https://github.com/home-assistant/micropython-home-assistant

So while MQTT works too but it is more complex and requires depencey of having an MQTT Broker so it is less user-friendly to end-users. There are howver a few integrations for Home Assistant that support both MQTT and WebSockets in order to be more flexible, I think that the main benefit with MQTT is that you could make a general implementation on the client side so not Home Assistant specific, in case you want to consider the use case of having integrations to other smart home ecosystems and platforms as well.

PS: Also note that "Local Push" is preferred over "Local Polling" to enable real-time state changes, see their IoT classifications here:

• https://www.home-assistant.io/blog/2016/02/12/classifying-the-internet-of-things/#classifiers

[jatgm1]

Hey I need to connect my moorebot scout (ros1melodic) to my smart home to get images (Preferably in color) and send commands to and get data from to use in automatons.

Sorry, I don't support EOL versions of ROS.

Tip for directly connecting Zigbee based Philips Hue lights to Home Assistant is to buy a Silicon Labs EFR32MG21 (or EFR32MG13) based USB adapter (like Elelabs/Popp Zigbee USB stick) or a Texas Instruments CC2652P/CC2652R based Zigbee USB adapter (like "Itead Sonoff Zigbee 3.0 USB Dongle Plus") and connect that via a long external USB extension cable (to get it away from any electonics so they do not interfer with signals) and then use Home Assistant's native ZHA integration component -> https://www.home-assistant.io/integrations/zha/

I have renewed interest in connecting Hue to OASIS. I'll start looking for solutions again.

Oh shoot that stinks. Idk why it's so hard to find a little robot with a charging dock that I can integrate into the home assistant. All the robots I find are usually closed source and require some app. The appbot riley, the robo buddy, the moorebot scout has some stuff available but it's operating system is apparently ancient and I can't even install packages because the repositories don't exist, so I can't figure out how to add the sources list to allow the download of an mqtt client to connect to my home assistant mqtt broker. And I don't know if I'm knowledgeable enough to install it from the source and ensure I have the dependencies and won't accidentally brick my device or something stupid. It's unfortunate that the ones with ros2 pre installed have no built in charging contacts and charging dock for completely autonomous use. U know drive it, hit return, have it automatically return (like the scout) and integrate into the other smart technology. I really need to learn to code in c.

[eigendude]

I wrote my own MQTT <-> ROS 2 bridge: https://github.com/eigendude/OASIS/blob/main/oasis_hass/oasis_hass/nodes/hass_bridge_node.py

The file automations.yaml file is entirely AI generated.

It works for my smart bulbs, but I think I can make it more generic using https://github.com/ika-rwth-aachen/mqtt_client. Home Assistant yaml is easy to generate with AI, so I can move more of the complexity into generated yaml.

[eigendude]

I've given up on mqtt_client. I could use it to translate from MQTT to ROS messages to avoid an MQTT bridge, but I'd still need a ROS bridge to house the logic of processing HA MQTT Statestreams, and probably automation events in the other direction. As a result, I'd have two bridges, one for mqtt_client (MQTT to ROS) and one for home automation logic (ROS to ROS). If both nodes were in Python, it'd add a lot of latency. So if I had to start over, I'd do a C++ nodelet, with the mqtt_client bridge loaded in, and a ROS-to-ROS c++ nodelet doing the home automation logic. But I already have good Python code so I'll stick with my single bridge.

EDIT: I couldn't get mqtt_client to work as a component but I'm including it as a node, currently idle, but with the ability to add bridge comms later without the steep code requirement of the custom bridge.

[Hedda]

Believe that WebSocket API connection is usually prefer for real-time bi-direction connections to native Home Assistant integrations:

• https://www.home-assistant.io/integrations/websocket_api/

  • https://developers.home-assistant.io/docs/api/websocket/

Found these video tutorials on WebSocket API in Home Assistant via a quick search:

• https://www.youtube.com/watch?v=Qk9A0QbG5-I

  • https://quickstarts.postman.com/guide/home-assistant/index.html?index=..%2F..index#0

• https://www.youtube.com/watch?v=Q5k2lj2iskw

• https://www.youtube.com/watch?v=34yD15jmeEA

Note! They have an official "thick" JavaScript WebSocket client for Home Assistant that can maybe also be used as a reference(?):

https://github.com/home-assistant/home-assistant-js-websocket

REST (RESTful) is an alternative but understand that API is for more quick and dirty connections instead of a proper integration(?):

• https://www.home-assistant.io/integrations/rest/

  • https://developers.home-assistant.io/docs/api/rest/

Did you also look at their native WebSocket and/or REST APIs?