Report statistics about the node usage

[mishaschwartz]

Some initial thoughts on how to implement this:

• collect information about the usage of the node internally
  • this could be some component that parses logs from other components and summarizes
• report stats of node usage through the network manager
  • protect this route (allow access to specific users only)
• show status of the components in the node in public page (similar to canarie)
• show the current server load in some crude metric (that doesn't give away any security issues) on a public page
  • estimate run time of some heavy load process

[tlvu]

Ask @matprov I think he already has a working log parsing solution using Grafana Loki that is currently running for the CCDP stack. Maybe we can integrate his solution into PAVICS.

Otherwise I heard good comments about the ELK (Elasticsearch Logstash Kibana) stack as well.

[tlvu]

For standard resource usage (CPU, RAM, Disk, Network I/O, ...), we already have Prometheus + Grafana stack for each node, see https://github.com/bird-house/birdhouse-deploy/tree/master/birdhouse/components#monitoring

Missing aggregation if we want stats from multiple nodes.