Metrics for Multiple Subscription Protocol

[ErnestKz]

The PR of #119 implements the multiple subscription protocol.

However implementing metrics for the new protocol have been neglected, and one of the old metric endpoints is re-used and do not capture the semantic significance.

i.e:

icepeak/server/src/Icepeak/Server/WebsocketServer/MultiSubscription.hs

Lines 283 to 285 in 1e991b2

	onConnect :: Client -> IO ()
	onConnect client =
	Core.withCoreMetrics (clientCore client) Metrics.incrementSubscribers

When a new client creates a multi-subscription connection, icepeak counts it as a subscriber, whereas in reality it can create more than one subscription.

The metrics that icepeak records can be found here:

icepeak/server/src/Icepeak/Server/Metrics.hs

Lines 40 to 73 in 1e991b2

	createAndRegisterIcepeakMetrics :: IO IcepeakMetrics
	createAndRegisterIcepeakMetrics = IcepeakMetrics
	<$> register (vector ("method", "status") requestHistogram)
	-- TODO: the following line can be removed after dashboard has been updated to use icepeak_data_size_bytes
	<*> register (gauge (Info "icepeak_data_size" "Size of data file in bytes."))
	<*> register (gauge (Info "icepeak_data_size_bytes" "Size of data file in bytes."))
	<*> register (gauge (Info "icepeak_journal_size_bytes"
	"Size of journal file in bytes."))
	-- TODO: the following line can be removed after dashboard has been updated to use icepeak_data_size_bytes
	<*> register (counter (Info "icepeak_data_written" "Total number of bytes written so far."))
	<*> register (counter (Info "icepeak_data_written_bytes_total" "Total number of bytes written so far."))
	<*> register (counter (Info "icepeak_journal_written_bytes_total"
	"Total number of bytes written to the journal so far."))
	<*> register (gauge
	(Info "icepeak_subscriber_count" "Number of websocket subscriber connections."))
	<*> register (counter (Info "icepeak_internal_queue_items_added"
	"Total number of items added to the queue."))
	<*> register (counter (Info "icepeak_internal_queue_items_removed"
	"Total number of items removed from the queue."))
	<*> register (histogram (Info "icepeak_sync_duration" "Duration of a Sync command.")
	syncBuckets)
	<*> register (counter (Info "icepeak_internal_ws_queue_items_added"
	"Total number of items added to the WebSocket queue."))
	<*> register (counter (Info "icepeak_internal_ws_queue_items_removed"
	"Total number of items removed from the WebSocket queue."))
	<*> register (counter (Info "icepeak_internal_ws_queue_skipped_updates"
	"Total number of updates discarded from the WebSocket queue."))
	<*> register (counter (Info "icepeak_subscribers_skipped_updates_total"
	"Total number of updates that have not been sent to subscribers."))
	where
	requestHistogram = histogram (Info "http_request_duration_seconds"
	"Duration of HTTP requests since starting Icepeak.")
	defaultBuckets
	syncBuckets = exponentialBuckets 0.001 2 12

We should create a new set of metric endpoints which capture data in accordance to how the new protocol functions. This could include:

• Number of subscriptions made on MultiSub protocol
• Number of connections on the MultiSub protocol
• Number of subscription deadline timeouts
• Payload failures

We also want to disambiguate the existing icepeak_subscriber_count to be explicitly SingleSub protocol connection count. Luckily icepeak_subscriber_count is the only metric that is fully tied to the SingleSub semantics.