variable_mask - variable_screen per bridge

[BuxtonCalvin]

I have PPG 1.1.10 working in docker without error using my EG4 18kpv with a POE tcp modbus bridge as source. I am using the influxdb ppg.bridge with newly enforced floating point values and everything seems to work fine.

I would like to limit the data extracted from the inverter to minimize modbus overhead, and limit the data sent to both MQTT and influxdb using variable filters. The change I'm asking for is to allow different variable filters for different bridges. My situation is that I would like to limit the MQTT transport to a handful of values that would signal a problem with the inverter (allowing rapid notices), while the influxdb filter would focus on data that routinely changes during operation--data that would then be sent to grafana. There may be some small overlap with these two bridge filters.

I imagine that you can only use one or the other "variable_mask - variable_screen" per bridge--not both, but I didn't see any notes on this. Perhaps make just one filter file name (per bridge) with a different .suffix ending to denote which filter approach is being used.

I would imagine a startup routine that would concatenate the various config file bridge filters into one polling filter of unique variables that would be used to poll the inverter, and then on the output side, the individual filters would condition the data flow to the various bridged outputs.

Any chance that this can done? Also, are the mask and screen lists delimited, columnar, both? Thx

[HotNoob]

hmm, i could see how that could be useful...
there isnt a dedicated feature for this... buuutt...

instead of var masks / screens, you can use the custom protocol feature.
https://github.com/HotNoob/PythonProtocolGateway/blob/main/documentation/usage/protocols.md#custom--editing-protocols

copy the eg4_v58 csv files:
https://github.com/HotNoob/PythonProtocolGateway/tree/main/protocols/eg4

into:

eg4_v58**.custom**.input_registry_map.csv
eg4_v58**.custom**.holding_registry_map.csv
eg4_v58**.custom**.json

edit your "custom" version, to exclude / include what you want.

then change your config per transport

protocol_version = eg4_v58.custom

as long as it has .custom in it, git will ignore it.
can be named whatever you want, ie: eg4_v58.influxdb.custom

[BuxtonCalvin]

OK, but that seems to only apply to the modbus poll. How would I handle the output side where I would further filter register variables per output bridge.

Or will PPG re-poll the inverter for each bridge if I set a different protocol for mqtt transport and influxdb transport. Trying to limit modbus polling here to the bare minimum.

[HotNoob]

[transport.slow]
protocol_version = eg4_v58.mqtt.custom
address = 1
port = {{serial port, likely /dev/ttyUSB0}}

#the 'transport' that we want to share this with
bridge = transport.mqtt

read_interval = 100

[transport.fast]
protocol_version = eg4_v58.influxdb.custom
address = 1
port = {{serial port, likely /dev/ttyUSB0}}

#the 'transport' that we want to share this with
bridge = transport.influxdb

read_interval = 10

[transport.influxdb]
type = influxdb_out
host = localhost
port = 8086
database = solar
measurement = device_data

[transport.mqtt]
#connect mqtt
transport=mqtt
host = {{mqtt ip / host}}
port = 1883
user = {{mqtt username here}}
pass = {{mqtt password}}
base_topic = home/inverter/
error_topic = /error
json = false
discovery_enabled = true
discovery_topic = homeassistant

like so. ish.

yeah if you have overlapping variables, they'd be polled twice.

[BuxtonCalvin]

OK will give it a try.

[BuxtonCalvin]

Sorry to bother you on this but I can't get two transports to work in tandem. If I comment out one or the other transport, then things work as they should. But if both are running, I get "cannot find attribute" errors. Running in docker with all six csvs (three for each transport) mapped to volumes.

Here's the config:
[general]
log_level = DEBUG

[transport.modbus_tcp]
log_level = DEBUG
transport = modbus_tcp
protocol_version = eg4_18kpv
analyze_protocol = false
write = false
host = 10.17.2.65
port = 502
bridge = transport.influxdb
manufacturer = EG4
model = 18KPV
serial_number = 4066670074
read_interval = 10

[transport.slow]
log_level = DEBUG
transport = modbus_tcp
protocol_version = eg4_18kpv.custom
analyze_protocol = false
write = false
host = xx.xx.xx.xx
port = 502
bridge = transport.mqtt
manufacturer = EG4
model = 18KPV
serial_number = 4066670074
read_interval = 60

[transport.mqtt]
transport=mqtt
host = xx.xx.xx.xx
port = 1883
user = xxxxcc
pass = xxxxggg
base_topic = inverter
error_topic = /error
json = false
discovery_enabled = true
discovery_topic = solar

[transport.influxdb]
transport = influxdb_out
host = 10.17.2.42
port = 8086
database = xxxss
username = xxxxcc
password = xxxxggg
measurement = device_data
include_timestamp = true
include_device_info = true
batch_size = 100
batch_timeout = 10.0
log_level = DEBUG

[HotNoob]

hmmm.

i havent personally tested the influxdb_out transport. not sure if its done right.
also havent tried two modbus_tcp transports on the same connection if that's the case?

i'll have to find some time to get a test setup going to check it out

[HotNoob]

#110

alright, i dove too deep into the rabbit hole.

influxdb support needs to be greatly improved.
ie, homeassistant's influxdb add on runs v1.8 and by default only uses v2.0 api 😢

so i'll have to setup a v1 influxdb and v2 influxdb to test with...

[BuxtonCalvin]

Yes, a lot of users stay with 1.x because of its simplicity and the API (which changed in 2.x).

Both transports work without error when deployed on their own. It's only when I run them in tandem, on the same mod bus device, that I get errors.

[HotNoob]

oh...

05c3917

haha...

early return so it never initialized the base transport. oops :P

that might fix it.

[BuxtonCalvin]

That fixed the error and now both transports load, however, while the MQTT transport is working as expected, the influxdb transport is showing only the specific data fields that I chose to broadcast via MQTT-- which in my case consist of only two data fields. There are about 30 other fields that should be in the influxdb data.

[HotNoob]

i'm going to have to set a side some time in the future to really go through the influxdb implementation.
it'll have todo for now; im drowning in other things todo right now... it could be a few months.

you can try running a second instance of ppg in the meantime.

[BuxtonCalvin]

A second instance works, but kinda klugey.... Trying to minimize possible collisions with poll timing for each instance, but you never know..