Request for getting started and contributing guide

[balupton]

Getting some docs on now to run the project and setup what seems to be the user scripts would be helpful. As well as docs on any contributing guidelines, like how to run tests or anything.

[cohlexyz]

It's fairly simple

• Install Violentmonkey or Tampermonkey to use the user scripts
• Install rust
• Build and run the server
  • $ git clone https://github.com/jaw-sh/stream-nexus && cd stream-nexus
  • $ cargo run
• The dashboard, chat, and overlay should now be available at http://localhost:1350/dashboard, http://localhost:1350/chat and http://localhost:1350/overlay
• Open a stream on Kick, Rumble etc. and the messages should show up in the overlay and dashboard.

One thing to note is that the user scripts , dashboard and overlay use 127.0.0.2 for connecting to the websocket server, you'll most likely have to change that to 127.0.0.1.

Alternatively, instead of using the user scripts you can also send dummy messages via Python:

import json
import time
import websocket
import uuid

websocket_server = "ws://localhost:1350/chat.ws"
ws = websocket.create_connection(websocket_server)

message = {
    "id": uuid.uuid4().hex,
    "platform": "Kick",
    "username": "Test",
    "message": "Test",
    "sent_at": int(time.time() * 1000),  # System timestamp for display ordering.
    "received_at": int(time.time() * 1000),  # Local timestamp for management.
    "avatar": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
    "is_premium": False,
    "amount": 0,
    "currency": "USD",
    "is_verified": False,
    "is_sub": False,
    "is_mod": False,
    "is_owner": False,
    "is_staff": False,
    "emojis": [],
}

ws.send(json.dumps({"platform": "Kick", "messages": [message]}))

[TheRenegadist]

Will all of this be required even post-completion or will there be a more accessible way to get this up and running without rust or any external software?

[jaw-sh]

This has been a personal project for my streams for a long time. The two people contributing to it are listeners. It runs without external software, unless you mean Rust for compiling or TamperMonkey for the scripts. The scripts are necessary otherwise there is no way to harvest data from the chats. All platforms (YT, Twitch, Kick, Rumble) try to block bots trying to read chat messages.

[TheRenegadist]

Got it, I don't have any experience with scripts, TamperMonkey, Rust, really any of this so I'm coming in pretty blind, I guess you could say I represent the average streamer. I'll mess with this when I can and run some tests next week.

Is Rust 100% required or can I run the scripts with TamperMonkey and get this going without it and just download the source as a .zip?

[y-a-t-s]

You'll need rust to compile the program unless you get someone else to do it for you.

[TheRenegadist]

Won't be necessary, I'll install Rust as well when I can. Appreciate guide and info.

[balupton]

To be clear on the instructions, one would need to install the tampermonkey extension in their own web browser and open the chats in their own web browser?

Considering we trust the rust program enough to run it anyway, couldn't the rust program open up all the chats in headful web views for login, and once logged in, make the web views headless, with the rust program injecting the user scripts itself into the web views. Or is such things detected and rejected by the streaming platforms?

[jaw-sh]

No. If you look at the changelog you will see attempts at running headless chromium. It is insufficient for a number of platforms to try and do so. YouTube in particular, which I imagine is what almost everyone will be using this on, does everything it can to break headless chrome.

[TheRenegadist]

I've installed Rust and TamperMonkey via Firefox but I'm not understanding how or where you run the $ commands, any advice for someone who has barely ever used command line?

Edit: Is there another place I can ask for help that doesn't bog down this issue?

[y-a-t-s]

It depends on what OS you're using. If it's Windows, then you will probably want to use Command Prompt or Powershell to run those commands. If it's Mac or Linux, then use the terminal.

Check out the first chapter of the Rust book to learn the basics and how to get things running with cargo. The later chapters are for teaching the language itself, but that first chapter can show you how to install and compile everything.

If you have any other questions, there are plenty of good Q&A resources online for Rust, and you can always ask on the Programming Thread on the Farms as well.

[TheRenegadist]

Well I got everything installed, I managed to get the stream-nexus server running, but it's not scraping any chats even after changing the websocket to 127.0.0.1 as instructed. The server is recognizing when I connect to the Dashboard with "New client connected to chat" but that's as far as it's going.

Edit: Here's the full powershell window after cd'ing to stream-nexus and then cargo run, this is being done on Windows 10 via Firefox for the user scripts:

  --> src\exchange.rs:12:5
   |
10 |     return Ok(serde_json::from_str(RATE_API_FALLBACK)?);
   |     --------------------------------------------------- any code following this expression is unreachable
11 |     // Send an HTTP GET request to the URL
12 |     let response = reqwest::get(RATE_API_ENDPOINT).await?;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ unreachable statement
   |
   = note: `#[warn(unreachable_code)]` on by default

warning: field `super_chats` is never read
  --> src\web\mod.rs:32:5
   |
31 | struct OverlayTemplate {
   |        --------------- field in this struct
32 |     super_chats: Vec<crate::message::Message>,
   |     ^^^^^^^^^^^
   |
   = note: `#[warn(dead_code)]` on by default

warning: field `last_command_at` is never read
  --> src\web\client.rs:21:9
   |
12 | pub struct ChatClient {
   |            ---------- field in this struct
...
21 |     pub last_command_at: Instant,
   |         ^^^^^^^^^^^^^^^

warning: field `id` is never read
  --> src\web\server.rs:10:9
   |
9  | pub struct Connection {
   |            ---------- field in this struct
10 |     pub id: usize,
   |         ^^

warning: `stream-nexus` (bin "stream-nexus") generated 4 warnings
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.32s
     Running `target\debug\stream-nexus.exe`
[2024-08-14T01:11:02Z INFO  stream_nexus::web::server] Chat actor starting up.
[2024-08-14T01:11:02Z INFO  actix_server::builder] starting 8 workers
[2024-08-14T01:11:02Z INFO  actix_server::server] Actix runtime found; starting in Actix runtime
[2024-08-14T01:11:19Z DEBUG stream_nexus::web::server] Sending 0 superchats.
Ok(HttpResponse { error: None, res:
Response HTTP/1.1 101 Switching Protocols
  headers:
    "upgrade": "websocket"
    "sec-websocket-accept": "HnOSrt0lEf/88nzOw+Em7Yrkr/c="
  body: Stream
 })
[2024-08-14T01:11:19Z DEBUG stream_nexus::web::server] New client connected to chat.

[cohlexyz]

Which site are you trying to get chat messages from? Kick, Odysee and Rumble should work, not sure about the other ones though.

[y-a-t-s]

YouTube in particular, which I imagine is what almost everyone will be using this on, does everything it can to break headless chrome.

Speaking of youtube, I noticed that yt-dlp is able to rip stream chats to a json file in real-time. It doesn't require any special auth cookies to do so either, last I checked. It could be worth looking into how they pull it off.