-
Notifications
You must be signed in to change notification settings - Fork 4
Research notes
Smart IP cameras connected to the hub, enabling various smart home video streaming features.
This real time video streaming feature is rare in design as it is based around our special RabbitMQ pipeline, hence the need for research, discussions, tests and planning.
This feature and related ideas were fully introduced and presented in Release 2 presentation document
https://docs.google.com/document/d/1zRQgcT4gpQFOTyjHJvwwUZjH11A9iRfbOsDnOX_-RIs/edit?usp=sharing
- Pivot due to risk analysis and imbalance user stories efforts/points after new requirements were provided.
- Demands for differentiation/expansion from stakeholders and possible startup incubators & shareholders
- Longevity, flexibility, scalability and sub-features growth of project
- Reasons like targeted clientele, funding, available hardware, reusability, and cost of deployment
For history and specific dates, please refer to Meeting Minutes Wiki
| Concept/Feature | Description | Application in Our Scenario | Notes |
|---|---|---|---|
| Forward Error Correction (FEC) | A method to add redundant data to streams, allowing error recovery without retransmission. | Improve stream reliability during packet loss or poor network conditions. | Ensures smoother playback at the cost of additional bandwidth. |
| Scalable Video Coding (SVC) | A codec enhancement that allows encoding a video at multiple resolutions, framerates, or quality layers. | Serve different users based on their device capabilities or network conditions. | Reduces computational overhead on the frontend; relies on backend for adaptive delivery. |
| STOMP and WEBSTOMP | Lightweight messaging protocols over WebSockets for real-time communication. | Use for signaling, user notifications, real-time control commands, and multi-user coordination. | RabbitMQ supports STOMP natively for seamless integration. |
| CMAF (Common Media Application Format) | A format optimized for low-latency video streaming by unifying HLS and DASH. | Enable compatibility with both HLS and DASH, reducing latency for LL-HLS. | Supported by most modern browsers and CDNs. |
| Group of Pictures (GOP) | A sequence of frames including one I-frame (keyframe) and several P/B frames. | Use small GOP sizes for low-latency streaming and fast ABR transitions. | Smaller GOPs reduce latency but may increase bandwidth. |
| Adaptive Bitrate (ABR) Options | Techniques to adjust video quality based on network conditions (e.g., bitrate switching). | Implement ABR on the Go Consumer or dynamically on the frontend based on client feedback. | Options include server-side ABR (e.g., DASH/HLS manifests) or client-side decisions. |
| RabbitMQ Queue vs Stream | Queues are point-to-point messaging with single delivery; streams allow replaying and high-throughput messaging. | Use streams for replayable and high-throughput scenarios like video segment storage; queues for transient control messages or events. | Streams may introduce more complexity; queues are simpler but transient. |
| Codecs (e.g., H.264, VP9) | Compression algorithms to encode and decode video streams. | Use H.264 for wide compatibility or VP9/AV1 for higher efficiency at the cost of encoding complexity. | Codec choice affects latency, compatibility, and bandwidth requirements. |
| Protocol/Format | Description | Advantages | Drawbacks |
|---|---|---|---|
| HLS (HTTP Live Streaming) | Breaks video into small chunks for streaming over HTTP. | Widely supported; CDN-friendly; easy to implement. | High latency due to chunk buffering. |
| LL-HLS (Low Latency HLS) | A variant of HLS for low-latency streaming with reduced chunk sizes and CMAF. | Achieves latency <3 seconds; integrates with existing HLS. | Complexity in configuration and CDN support. |
| DASH (Dynamic Adaptive Streaming) | Similar to HLS but uses XML manifests and supports more ABR strategies. | Highly configurable; codec-agnostic. | Browser support is less universal compared to HLS. |
| WebRTC | Peer-to-peer real-time communication protocol for ultra-low-latency streams. | Sub-second latency; direct client-to-client connection. | Complex signaling; bandwidth scaling for many users. |
| Feature | RabbitMQ Queues | RabbitMQ Streams |
|---|---|---|
| Message Delivery | One-time, point-to-point. | Replayable; multiple consumers can access past messages. |
| Throughput | Moderate. | High throughput; suitable for large-scale video segments. |
| Latency | Low latency for transient messages. | Slightly higher latency due to replayability, but lower for high volume |
| Use Case | Control commands, transient events. | Video segment storage, replayable event logs. |
|
| Persistence | Requires durable queues | Built-in, efficient persistence
|
| Scalability | Scaling queues requires extra complexity | Horizontal scaling with partitions
|
| Multiple Consumers | Competing consumers share the same queue | Independent offsets for each consumer
| Technique | Description | How to Apply in Your Pipeline |
|---|---|---|
| Forward Error Correction (FEC) | Adds redundant data for error recovery. | Use in Go Consumer to encode streams for packet loss scenarios. |
| Retransmission Strategies | Requests missing packets to recover from errors. | Employ selectively in Go Consumer for critical video segments; avoid in ultra-low-latency scenarios. |
| ABR Transitions (Client-Side) | Client adapts playback quality based on network conditions. | Implement in frontend using HLS/DASH manifests with multiple quality levels. |
| SVC (Server-Side ABR) | Streams with multiple layers for different quality levels. | Serve SVC streams directly from the Go Consumer based on user requirements. |
| Scenario | Recommended Technology/Technique |
|---|---|
| Server Downtime | RabbitMQ Streams for message persistence; CMAF for smooth playback. |
| Bad Network Conditions | ABR with FEC or lower bitrate GOP transitions; client-side SVC decoding. |
| Multiple Users, Multiple Cameras | SVC streams to serve users with different devices/network conditions. |
| Multiple Users, Same Camera | RabbitMQ fanout exchanges for efficient stream distribution; STOMP for user management and signaling. |
-
This sub feature is the most recent addition upon discussion with the stakeholders in sprint 10. With the video streaming feature making good progress, motion detection will be implemented in the form of alerts. The motion detection logic is to be implemented on the hub, and motion alerts will be sent to the client(web instances/Supabase) over RabbitMQ.
-
Stakeholders asked for a way to enable and disable motion detection. For example, detected motion will not generate an alert if the tenant is home, it will only generate an alert in "away mode" (temporary name).
-
Research was conducted to determine the best way to implement this feature with our current project structure.
Research notes are in the Research Document (same as linked at the top of the wiki).