Running multiple groups in Griduniverse from the same Heroku instance: 13pts

[nataliavelez]

Description of the problem: We cannot set up Griduniverse to run (a) multiple groups from the same instance, and (b) multiple generations. We cannot use any network settings from Dallinger, or take advantage of the Dallinger functionality that allows for large-scale multiplayer, multi-generational studies.

• We cannot run multiple iterations of the game from the same instance. That means we need to create a separate game instance / Prolific experiment / Heroku app for every single group we collect data from. This is a very limiting problem, as it dramatically increases the cost and rate of attrition of each study, making it impractical to collect data with Griduniverse at all.
• We cannot change the network to allow for multi-generational studies. When we try to change the network to a Chain or Discrete Generational network, it changes in the database but not the server itself (you still see all of the players on the screen, and the next generation can’t spawn afterwards).

We have tried to address these problems ourselves by commenting out create_network and create_node, but nothing changes in the game. We can’t change these functions or figure out if they are actually being called.

We think this issue stems from how Griduniverse is set up as a Fully Connected network internally in order to allow concurrency.

Proposed solution: We think that these issues stem from the same source - of these, the highest priority is allowing Griduniverse to run multiple iterations of the game from the same instance. That means that, as the experimenter, I could specify that I'd like to recruit, e.g., N = 60 participants in groups of 4 and that, as the participant, I would be assigned to whichever of 15 groups has an opening available.

Notes from brainstorming session

1. Dallinger's waiting room implementation will need to support adding participants to multiple networks, if it doesn't already
2. The code that manages game state (scores, players, time left, etc) and the game event loop is currently part of the Experiment class. This makes it impossible to create multiple games within an experiment run. We need to separate game state and event loop management, so that the experiment can create multiple games within a single experiment run.
3. In order to support 2., experiment setup needs to assign a different griduniverse_ctrl and griduniverse style websocket channels to each of these new GameState instances.

[nataliavelez]

Thanks to @emieczkowski for first describing this issue - any omissions/errors my own

[jacobyn]

Looking into the code, it does not seem like you have a network structure in your experiment. How you would imagine the logic work here in term of Dallinger? I can't see a sense of iteration in the code.

[nataliavelez]

Griduniverse does have an underlying network structure - by default, this network is fully connected to allow concurrency. Elizabeth and Bonan have tried but not yet been able to change the network structure themselves.

My best guess is that this issue could be resolved by adding a custom network structure. To follow up on the example above: It'd be great if we could specify a PartiallyConnected network with, e.g., N = 60 nodes split into disconnected subgraphs of m = 4 participants. Each subgraph would be equivalent to the N = 4 FullyConnected graphs that we're currently using when we run each group in a separate instance.

But there are some subtle details here, and I'm not sure if Griduniverse is set up to support them - for example, the game should start running when each m = 4 subgraph is filled, and not when all N spots are accounted for. I'm hoping to get more clarity about how complex it would be to implement something like this when we meet tomorrow.

[alecpm]

In terms of the network structure, there may be an alternative to creating a new partially connected network type.

Dallinger experiment instances can have multiple networks. The default behavior is for each participant to be assigned randomly to a network with available space and then join each network sequentially. The variable defining the number of networks is named experiment_repeats for this reason. However, I think an experiment could customize get_network_for_participant to provide different behavior, allowing for all the networks to be filled at once with different participants.

[nataliavelez]

That sounds like a good solution - thank you, Alec!

[alecpm]

@nataliavelez I also wonder if the "multiple generations" aspect could be accomplished using the existing num_rounds and time_per_round configuration. GU doesn't really make use of Dallinger network structure (e.g. no vectors, transmissions, etc.) and modifying it to make use of network structure could be a pretty big undertaking.

[nataliavelez]

@alecpm That's great to know! So, to clarify: Does that mean that we don't necessarily need to modify the network structure, either to run multiple groups simultaneously or to do multigenerational designs? Because that would be great - the network structures idea was my best guess about how to approach this problem in base Dallinger, but I'm not tied to that implementation.

Would it make sense to split up these features into two, parallelizable stories? e.g.,

• Allowing multiple groups in the same instance by customizing the behavior of get_network_for_participant
• Support for multigenerational designs using the num_rounds and time_per_round configuration

[alecpm]

@nataliavelez I think this issue can be treated as the first story. A second story probably makes sense once the details regarding how a multigenerational design would differ from a simple multi-round version of the game (which is already supported).