networkDynamic for panel of larger networks

[ThomasKraft]

In the current form, the function networkDynamic() is prohibitively slow for converting a list of panel networks of moderate to large size to a dynamicNetwork object (for example, a list of 2 networks with ~8000 nodes and 16k edges is taking on the order of hours). As far as I can tell, this issue stems from the implementation of a for-loop through all edges of the network here:

networkDynamic/R/utilities.R

Line 628 in 66e66b9

for (i in seq_len(nrow(edge.data))) {

.

Based on the comments in the code, I am wondering if tergm or anything else already has a different way of taking on this task that is more efficient. I'm having trouble figuring out where exactly tergm does this, but I think it must do something similar when simulating. Perhaps an operation performed on a stack of sparse sociomatrices would be applicable here?

[martinamorris]

@chad-klumb given your recent work on tergm and tergmLite do you have any thoughts on this?

[chad-klumb]

I would be surprised if tergmLite really required storing a network list (in typical use cases).

Looping over (a large number of) inputs and growing the output at each iteration can be slow as growing often means copying (in R). It is sometimes possible to rewrite the code in such a way that repeated copying is avoided.

tergm applies changes to the networkDynamic using this function:

https://github.com/statnet/tergm/blob/b13a3a47c778eb325233073be62795fa5dd14309/R/stergm.utils.R#L153-L232

[martinamorris]

@ThomasKraft are you using networkDynamic outside of the tergm or tergmLite contexts? e.g. for network movies via ndtv?

[ThomasKraft]

I am using networkDynamic outside of the tergm context, specifically working with a list of networks that are output from EpiModel simulations with duration = 1 (which unlike longer durations does not store a networkDynamic object). Converting to networkDynamic class is useful both for creating movies with ndtv as well as analyzing distributions of tie duration and other dynamic network properties.

From the link @chad-klumb sent, I see now why the tergm case is different than the post-hoc conversion of the list of networks. So it seems like I'd need to tweak the networkDynamic code originally linked for it to be more efficient?