proposal: simplify database configuration

[dannyrandall]

The most complex part of the av-control-api as it stands is the database configuration. There is a lot of overhead that we've introduced from our early SQL migration as well as legacy configuration items would either no longer use, or thought we would need but don't. I think that it would make it much easier for other organizations to use and configure their systems if we take the time to simplify the configuration by removing unnecessary pieces.

I'll go through each of the databases and note all of the changes that could be made to each of them.

Buildings

There is zero useful info in the buildings database. Remove it.

Room Configurations

This is a notoriously confusing database. Its function is to make it possible to pick specific command evaluators for a given room. In practice, we have written command evaluators that will work in all cases, and just use the same room configuration in every single room. I think that removing this database and "baking" the command evaluators to run into the API itself would make the configuration much easier to understand - the theoretical loss in picking which command evaluators to use would require smarter command evaluators in some cases, which I think is a worthwhile tradeoff.

Rooms

• Remove the configurationID block since it will no longer be needed with the removal of room configurations.
• The name field can be removed; only the _id field should be used for identifiying the document.
• Add a tags block - a map of string to interface{} (string, bool, number, or object). If someone wants the ability to filter their rooms in some manner, you could do so with tags. We could add predetermined tags into the config tool (like notes below) if we wanted specific tags to be filled out by the installers or support.
• Remove description. notes can be added to tags to match what it would be used for in the config tool.

{
  "_id": "ASB-A343",
  "designation": "production",
  "tags": {
      "notes": "- Has a light in the ceiling\n - Secret door behind the podium",
      "capacity": 50,
      "roomType": "MMC", // or podium, etc
      "activeClassroom": true
  }
}

Device Types

• Remove the description field
• Remove the display_name field
• Remove the source, destination, and tags fields
• Remove ports - it's only being used for our config tool right now
• Remove roles: Its functionality can be met through commands - e.g., if there is a command to GetBlank, then that should be reflected in the API response. To figure out which devices should be shown in the API response (roles VideoOut, AudioIn, etc), we simply build the graph for both audio & video and know that the devices at the end of the graph are devices that should be in the response.
• commands
  • becomes a map of CommandID to an object that includes the following fields:
    • address: the full URL of request for the av-api to send
      • Should be written using template syntax, with the variable names matching what is "passed" to the driver interfaces
    • order: if multiple actions are generated at the same time, this defines the order they should be executed in. If order is missing, the command is placed after all order'd commands
  • remove description
  • remove microservice object
  • remove endpoint object

{
  "_id": "Atlona6x2",
  "commands": {
  // this is a map because I can't think of a reason to have multiple of
  // a specific command. If there is a reason, this can be and array
  // of objects.
    "ChangeInput": {
        "address": "http://localhost:8026/{{.Address}}/output/{{.Output}}/input/{{.Input}}/6x2",
        "order": 10
    },
    "InputStatus": {
        "address": "http://localhost:8026/{{.Address}}/ouput/{{.Port}}/6x2",
        "order": 10
    }
  }
}

Devices

• Remove display_name - this belongs in the UI configuration.
• Remove description. notes can be added in tags like in Rooms.
• Change type._id to just be typeID - it still maps back to a device type
• Remove roles - see the above reasons on device types.
• Proxy remains the same
• ports
  • _id changes to name
  • remove friendly_name, description, source_device, destination_device
  • add source - a bool indicating that the traffic "originates" from this device
  • add type - a string indicating the type of "traffic" going to/from the port
  • add endpoint - the ID of the device connected to the other side of the port
• Change tags to match Room tags

{
    "_id": "TNRB-240-SW1",
    "address": "10.66.51.242", // or a hostname here
    "typeID": "Atlona6x2",
    "proxy": {
        ".*": "TNRB-240-CP1.byu.edu"
    },
    // we do have cases where multiple ports with the same name
    // exist, so this should be an array - not a map
    "ports": [
        {
    	    "name": "IN1",
    	    "endpoint": "ITB-1101-VIA1",
    	    "source": false,
            "type": "video"
        },
        {
            "_id": "OUT33",
            "endpoint": "TNRB-240-D1",
            "source": true,
            "type": "video"
        }
    ],
    "tags": {
        "notes": "Located in the ceiling in TNRB 250",
        "color": "red",
        "speed": 300,
        "old": false
    }
}

UI Configuration

The database changes for UI Configuration will be addressed in a proposal in the UI repo

Making these changes will require updating the av-control-api command evaluators to use the new configuration objects, as well as any services that also use the configuration (via, shure, device-monitoring, BFF, etc). We would also want to write a script to go through items in the database and convert their existing data into this new format. I would expect this change to take my time and two students' time for around a month to get the core API and drivers updated. There would be additional time required to update SMEE depending on how far along we are with that tool, and some (around 1 sprint of time, one person) to update the BFF.

Please feel free to respond with any thoughts you have on how we can improve these changes.

[mjsmith3]

Having written the documentation for all of the database fields... I support this proposal. I especially like the use of "tags". It lets us tore useful information without making new fields for devices/rooms that don't have the info (or we don't want to track for whatever reason). I think there are some edge cases that need to be thought through for removing roles (i.e. mirroring rooms, rooms with special audio configurations etc.). I am sure there is a way to handle these edge cases with a universal command evaluator, but that is something I am not super familiar with. But overall simplifying the DB will make it easier not only for ourselves, but also easier for other universities as well.

[jpw547-zz]

• Room names - I can agree that the name field is almost entirely obsolete. I am mostly thinking about rooms like HFAC-D400 which is much more commonly known as the Madsen Recital Hall, or the Madsen, to name one example. This could be a part of the tags or attributes thing or whatever, but there are places around campus that are not known by official room numbers but instead known by some other name
• Device type display_name - I added this field for the sake of making the device types more readable in the builder of SMEE. So if everyone is fine with "DM-MD16x16" and knows what that means, then sure I guess we can get rid of it.
• Device type source & destination - these were added to signify what kinds of devices are valid options for the port source and destination device fields in the builder. So in a similar way to the other feedback I have, if you have an alternative solution to account for this information without these fields then that would be great.
• Roles - I agree that roles should just be strings, but I believe the roles on devices were there such that a customer could choose to have a TV or something where there is no audio control for it. Like if they want to mute it themselves or set the volume to zero, and then nobody can ever change that.
• Port source & destination - how do we know the path of the signal if the ports don't include what is connected to that port? It seems like we are removing the ability to show what is connected to what. ... I now see that this information is included on the device itself. I guess that is fine.

[dannyrandall]

@jpw547 My thoughts on the above:

• I think tags is a great place to put "unofficial" names of rooms.
• Like we talked about offline, we'll probably need a different database for SMEE configuration/template type items. I think our device type ID's should probably be sufficient (we can change ID's to be usable), but for things like default ports for a specific type, we'll need a separate DB for those items.
• The idea with source is that you'll have the information required to decide which options should be valid. For example, valid options for a TV's port would be the set of inputs and ports where source is true. However, I agree that it could be a little clearer.
• We could accomplish the same thing through a device type without the SetVolume and SetMute commands - like the Sony TV No Audio device type we have right now.
• Again, the source lets you know which way the signal is going. I agree that it could be more clear.

Based on feedback (@jpw547 as well as others, offline) here are some changes I'd like to make to the proposal:

• Remove source from ports. Instead, add incoming and outgoing booleans that default to false if they are not set. One benefit of this change is that we can appropriately describe ports that do have bidirectional data flow, like in the case of our AV-over-IP solutions. The port JSON will now look like:

{
    "_id": "OUT33",
    "endpoint": "TNRB-240-D1",
    "incoming": false, // not required
    "outgoing": true,
    "type": "video"
}

• Change commands to support richer HTTP requests. The config for a command would look like:

{
    "method": "GET",
    "address": "http://localhost:8026/{{.Address}}/6x2",
    "headers": {
        "Content-Type": "application/json"
    },
    "body": {
        "output": "{{.Output}}",
        "input": "{{.Input}}"
    }
    "order": 10
}

The defaults would be a GET request with an empty body and the default headers sent with the Go HTTP client. address is always required.

Questions (@bwinterton & @thailandmatt, I would love your feedback):

• Thoughts on incoming and outgoing on ports?
• Would it make sense to add tags to device types? Or does it make more sense the way it currently is, where tags can only be added to devices.