Enhancement: Better Remapping Capability

[Croq360]

The current remapping that's implemented, while useful and helpful, could be improved. My suggestion is supporting a way that users could have control over the channel numbers (and possibly displayed names), and the ability to deactivate channels you don't want to have available. Emby supports sorting channels by name in the guide, which is nice, but removing a channel from the guide is not very straight-forward, and if you ever change your listings, you have to go back in and do it again. Plex allows you to remove channels from the guide in their manage Live TV & DVR page, but in the guide, you can only sort by channel number, no other option.

By giving more control to users on channel numbers/names, and activating/deactivating channels, you could have one place to manage all that in locast2tuner.

Some people might ask, "Why not just use xTeVe?" I am actually using that now. The issue there is that if you want to have both Plex and Emby supported at once, you'll likely have to run two instances of xTeVe. The reason for this is xTeVe's buffer capability, the default is to just hand the original m3u stream over to the client (like Plex/Emby) to play. This works great for Emby, but Plex doesn't support that. You could switch to have xTeVe do the buffering, but then you need to configure Emby to use locast2tuner in HDHR mode so the buffering will work, otherwise you can't play streams in Emby. Or you set up two instances of xTeVe like I have done, one with buffering, one without. Point Plex at the buffering one and Emby at the non buffering one. The problem here is that now you have to manage two separate channel remappings.

My proposal is for locast2tuner to offer a remap JSON file that allows you to go in and edit to your liking, then reference that remap when you run locast2tuner to get the numbers/names you want. While it would be nice to be able to edit these values through some tool in locast2tuner, if that's not possible, then having a way to get a starter file would be good, something like going to localhost:6077/remap.json. Save that file off, then edit, save, and point locast2tuner to use your modified list of channels.

{
  "channel.1571434910068": {
    "original-name": "KTVK (Phoenix)",
    "remap-name": "AZ Family (Phoenix)",
    "original-number": "3.1",
    "remap-number": "3",
    "active": true
  },
  "channel.1571434910374": {
    "original-name": "CBS (Phoenix)",
    "remap-name": "CBS (Phoenix)",
    "original-number": "5.1",
    "remap-number": "5.1",
    "active": true
  },
  "channel.1571434911228": {
    "original-name": "PBS (Phoenix)",
    "remap-name": "PBS (Phoenix)",
    "original-number": "8.1",
    "remap-number": "9.1",
    "active": true
  },
  "channel.1571434911800": {
    "original-name": "FOX (Phoenix)",
    "remap-name": "FOX (Phoenix)",
    "original-number": "10.1",
    "remap-number": "7.1",
    "active": true
  },
  "channel.1571434912373": {
    "original-name": "NBC (Phoenix)",
    "remap-name": "NBC (Phoenix)",
    "original-number": "12.1",
    "remap-number": "6.1",
    "active": true
  },
  "channel.1571434913084": {
    "original-name": "ABC (Phoenix)",
    "remap-name": "ABC (Phoenix)",
    "original-number": "15.1",
    "remap-number": "4.1",
    "active": true
  },
  "channel.1571434913506": {
    "original-name": "COURT (Phoenix)",
    "remap-name": "COURT (Phoenix)",
    "original-number": "15.4",
    "remap-number": "145.1",
    "active": false
  },
  "channel.1571434916392": {
    "original-name": "CW (Phoenix)",
    "remap-name": "CW (Phoenix)",
    "original-number": "61.1",
    "remap-number": "8.1",
    "active": true
  }
}

In the above example, I have disabled Court TV. I have renamed KTVK to "AZ Family" (how it is better known here). And I have renumbered ABC, CBS, NBC, FOX, CW, and PBS to 4.1, 5.1, 6.1, 7.1, 8.1, and 9.1 respectively. That gets AZ Family up top where I want it, followed by all of the network channels, and I could renumber all the remaining channels to my liking as well.

The key for each entry in the JSON data is the unique channel id for that channel (which should not be changed).

Regarding the "remap-name" and "remap-number", maybe to make things easier, they just should stay empty unless you actually want to change something (e.g., if you did this, then the entry for CW would have "remap-name": "", and locast2tuner would not rename it in that case). Just an idea.

In the event any new channels were to come along, they'd just default to no rename/renumber and be put wherever they happen to appear. Perhaps having the logging flag that could key a user to know that they have some new channels to deal with. Not sure how to deal with channels getting removed, I assume that happens. Also, I don't know if Locast would ever go through and change the channel id's, but I guess if they did, the user would have to deal with those too.

I don't know whether the current remapping process should remain (in case you have users that just want that simplicity), or if this new process replaces the current one.

[Croq360]

Last night, I was tinkering around with how to render the remapping info on a page where you could edit it. I came up with rough prototype, just to see if it could work.

I found a package called Tabular (tabular.info), downloaded it, created the attached HTML file and put it in the root of the Tabular folder. If you do that, you can load the HTML file take my snippet above and save that to a JSON file, then load that from the webpage (Choose File, then Import), you'll see a table rendered that you can edit. Every column can be sorted on and all columns are editable except for channel ID.

Notes:

• The JavaScript code creates a local array of dictionaries to feed into Tabular. The Tabular package couldn't make sense of what is above.
• There is no code in there to save the JSON to a file (another transformation back to the format above would be needed before you do that) - it was way too late at night to try and tackle that.
• This approach is more intended to be an inspiration or starting point. Whether it's ultimately included in the web interface for locast2tuner (which seems like the logical place to put it, in my opinion), or winds up being a separate tool is to be determined.
• The attached file has a TXT extension - change it to HTML to try it out.

test.txt

[Croq360]

I'll see if I can get some time to tinker some more on this tonight and at least get it set up to support the full round trip (Read in JSON->HTML edit->Write out JSON). I can work on "prettying it up" after that... :)

[Croq360]

@wouterdebie - for this approach, are you thinking the editing of the remapping data would be something you could do inside of the main web interface (like 127.0.0.1:6077/remap, have that load your current remapping, if any, and you could edit it). The tricky part might be on how to save the remap data once you're done. Also, do you think using the TOML format would be better or sticking with something like JSON?

In order to use the tabulator features, you need to feed it an array of dictionaries, so you'd need to change the above structure to be an array, and put the channel ID in each of the dictionaries, something like "channel-id": "channel.12345". Then tabulator can display it quite nicely and allow you to edit to your liking. Once you're done, it'd need to convert back to the original format and then save that to JSON (or TOML) file.

Alternatively, the remap format we use could already be an array of dictionaries. When you read in that data, you do a quick lambda expression to convert it to the format above to make it easier to work with in the actual remapping that locast2tuner would do. Going that route would make the remap edit process easier, but you'd have the bit more complication in locast2tuner to do that conversion as you start your remap processing.

For example, in Python, let's say I had an array of dictionaries in a variable called remap. The structure of each item is something like this:

[
  {
    "channel-id": "channel.1571434910068",
    "original-name": "KTVK (Phoenix)",
    "remap-name": "AZ Family (Phoenix)",
    "original-number": "3.1",
    "remap-number": "3",
    "active": true
  }
]

To get that into the format that I originally posted, dictionary of dictionaries, with a key of the channel id, I could just put it into another variable using a call like this:

remapDict = {d["channel-id"]: d for d in remap}

Note that you'd still have the "channel-id" item in each dictionary, but that's probably okay for your processing to apply the remaps.

{
  "channel.1571434910068": {
    "channel-id": "channel.1571434910068",
    "original-name": "KTVK (Phoenix)",
    "remap-name": "AZ Family (Phoenix)",
    "original-number": "3.1",
    "remap-number": "3",
    "active": true
  }
}

[Croq360]

Okay, I have put together a bit more functional version of the remapping editor. It should at least give you an idea of what it can do, but there are more things I want to try out (see below).

As a reminder, you need to download the tabulator package from tabulator.info, then save the attached remapEditor.txt as remapEditor.html in the root folder for tabulator.

You would need the channel remapping information in the format in my first post. Paste that into the JSON data area.

Click on Copy to Table to create the tabulator table on the right. Note that you can reorder on any of those table columns, but only the remapped name/number and active flag are changeable for each item.

Once you've made your edits, click Copy to JSON to copy your edits into that JSON data area.

At this point, you could copy the text in that JSON window and save those changes to remap.json (or something like that).

Note that obviously, locast2tuner doesn't support this yet. I'd like to try and figure out a way to load the JSON remapping data from a file you pick (should be easy to do), and then save your changes to that file (not sure if you can do that directly from an HTML/JavaScript page, you can certainly save a file, just not sure you'd be able to overwrite an existing file).

remapEditor.txt

[wouterdebie]

Cool! I'll start with just having a remapping file that you can specify on the command line and then think about how the UI should work. As you mention, one of the things is how to save data. Do we want this in just a json file or do we want to use some other storage mechanism?

[Croq360]

I was thinking about that too. Could locast2tuner manage the remapping internally and just use something similar to what I've put together to manage the remappings through the web interface?

I mean what I have provided can work, just more of a "manual" process to load and update the data than I'd like.

Though maybe that's not the best idea to keep it "internal" either. Having access to the JSON directly would allow you more ability to edit things if you wanted. Like doing a bulk change if you wanted to change "(Phoenix)" to "(AZ)" or something like that.

[wouterdebie]

@Croq360 What do you think in general the behavior should be? Do we keep --remap as well, or do we replace this with the remap file? In case we keep it, what would you expect to happen? Remap first with --remap and then apply the mapping from the file? Or do we only apply the file?

[Croq360]

Maybe we could leave the current approach in there, using "--remap" to tell locast2tuner that you want to remap. Then there "--remap-file=remap.json" (or whatever the right format is). If that's provided, then you're telling the program to remap based on the data in that JSON file instead of using the standard way. I suspect that there are at least some people that like the simplicity of the current approach, so I'd hate to take that away from them.

[wouterdebie]

@Croq360 cool. I just pushed a proof of concept. Would you mind checking this out? Right now --remap takes precedent over --remap_file. I also changed the format of the json a bit, since I wanted to keep using the correct terms. With --remap we map the channel number and the call_sign, so I'm referring to this here as well. The original_channel and original_call_sign are purely for documentation and unused in the code.
Lastly, I removed the (city) from the remap, since that's added by the muliplexer as a last step, rather as part of the remap.

Let me know what you think and if this works as expected.

[Croq360]

Sure, I'll grab that and take a look. Any reason why I couldn't config the test version to run on a different port and keep my current version running? Assuming of course that I don't try to do too many streams at once... :-)

[Croq360]

I cloned the branch you created for the remap PoC, but when I went to build it with "cargo build", I got the following errors:

   Compiling unix_socket v0.5.0
error[E0433]: failed to resolve: could not find `unix` in `os`
  --> C:\Users\Rutabaga\.cargo\registry\src\github.com-1ecc6299db9ec823\unix_socket-0.5.0\src\lib.rs:19:14
   |
19 | use std::os::unix::ffi::OsStrExt;
   |              ^^^^ could not find `unix` in `os`

error[E0433]: failed to resolve: could not find `unix` in `os`
  --> C:\Users\Rutabaga\.cargo\registry\src\github.com-1ecc6299db9ec823\unix_socket-0.5.0\src\lib.rs:20:14

And then tons more errors (presumably related to the above). Any ideas what I'm missing?

Same results if I run with "cargo build --release". The website https://docs2.w3cub.com/rust/std/os/unix/ffi/trait.osstrext/ seems to suggest that maybe it's unix only?

[wouterdebie]

Ah yes.. It seems like slog-rs includes syslog which is unix only.. Let me see if I can find a way to omit that for windows.