RunetimeError: invalid multimodal distribution

[clairelaudy]

I am trying to run BioPathNet with the following config file (NB 'strict_negative' option is set to 'no') on some small learning and validation graphs.

running it with the attached files produces a RuntimeError.

However, if I remove the line
'role is_a biocypherroot' (line 7)
from the learning.txt file, with the same config.yaml file, BioPathNet runs without any error.

brg.txt

entity_names.txt

entity_types.txt

learning.txt

mockdata_run.yaml

test.txt

valid.txt

[emyyue]

I tried it out

1. using the files you provided. All works well for me.

2. Remove the line 7 in learning.txt: This is when I get a RunTime error

12:49:25   >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
12:49:25   Epoch 0 begin
12:49:25   >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
12:49:25   binary cross entropy: 0.700183
12:49:25   >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
12:49:25   Epoch 0 end
12:49:25   duration: 0.13 secs
12:49:25   speed: 23.06 batch / sec
12:49:25   ETA: 0.00 secs
12:49:25   ------------------------------
12:49:25   average binary cross entropy: 0.709188
12:49:25   Save checkpoint to model_epoch_1.pth
12:49:25   >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
12:49:25   Evaluate on valid
/Users/emyyue/miniforge3/envs/biopathnet/lib/python3.10/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/native/TensorShape.cpp:3484.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
Traceback (most recent call last):
  File "/Users/emyyue/Documents/github/BioPathNet/./script/run.py", line 100, in <module>
    train_and_validate(cfg, solver)
  File "/Users/emyyue/Documents/github/BioPathNet/./script/run.py", line 62, in train_and_validate
    metric = solver.evaluate("valid")
  File "/Users/emyyue/miniforge3/envs/biopathnet/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/Users/emyyue/miniforge3/envs/biopathnet/lib/python3.10/site-packages/torchdrug/core/engine.py", line 222, in evaluate
    metric = model.evaluate(pred, target)
  File "/Users/emyyue/Documents/github/BioPathNet/./biopathnet/task.py", line 260, in evaluate
    neg_t = functional.multinomial(prob[:,0,:], 1, replacement=True)
  File "/Users/emyyue/miniforge3/envs/biopathnet/lib/python3.10/site-packages/torchdrug/layers/functional/functional.py", line 17, in multinomial
    return torch.multinomial(input, num_sample, replacement)
RuntimeError: invalid multinomial distribution (sum of probabilities <= 0)

The problem is that when heterogeneous_negative: yes and is enabled, the code filters entities by type, and for some validation triples, all entities of the correct type are masked out (likely because they're all true positives). This leaves a probability distribution where all values are 0, causing the multinomial sampling to fail.

[emyyue]

My questions would be if it is important for you that negatives are sampled always from the same node type.

e.g. gene participates_in pathway, it is very much necessary to always sample other negative pathways, instead of genes as well (gene participates_in gene does not make sense)

And if you have a bigger graph where some negatives from the same node type can be sampled.

---

Node Type-Aware (NTA) negative sampling is heterogenous_negative, sorry we changed the names after some time

[clairelaudy]

From what I understand from your answer, during the learning phase, the sampling is done by taking a true positive, and building a negative by kipping the source (head) node, and finding a taget (tail) node from the ones in the graph, with the same type.
Therefore, when I remove one of the "x is_a biocypherroot" line, there is not enough nodes of type THING (such as biocypherrot) in order to build a negative triple.

My question then is, why does the error occur in the validation phase ?
Is it possible that you point me out the code that does the negative sampling ?

If my understanding of the negative sampling is rigth, I have another question. Using this method, you can never select a node that is alway a target node in the graph and never a source in order to build the negative samples. Is that not a problem for the learning ?

[emyyue]

1. Yes, there is basically NONE of that node type to build a negative triplet, bcs biocypherroot (type THING) has been excluded with heterogeneous_negatives: yes. If there is at least one more, than that would be sampled as neg. In this case, there is none at all.

2. During validation? Then negative samples must only be used for performance metrics... Here is the code

• for negative sampling during training is called here and then this is the strict_negative

  BioPathNet/biopathnet/task.py

  Line 340 in f40af00

  def _strict_negative(self, pos_h_index, pos_t_index, pos_r_index):

• during validation, this might be the problem. This is the only place (I can think of now) where multinomial used. If you don't care about AUPRC/AUROC, then I would suggest to entirely remove this part. Bcs usually in KGs we only care about ranking based metrics (MRR, Hits@k,..)
  

  BioPathNet/biopathnet/task.py

  Line 260 in f40af00

  neg_t = functional.multinomial(prob[:,0,:], 1, replacement=True)

3. I am not completely sure, I get it. It only depends on the node TYPE, e.g. when you have gene1 participates_in pathway1, for training to get the negative samples, you are right, it will perturb the target, when learning in forward direction. So that you might get a negative of the formula gene participates_in pathway. But node of type pathway could also have appeared as source pathway2 is_related_to pathway3. pathway2and pathway3 are both legitimate negative samples (if they were absent in the training graph - brg.txt and learning.txt)

[clairelaudy]

Thanks for your answer. I managed to run both learning and prediction phases on some bigger examples after removing the AUPRC and AUROC as you suggested.

Anyway, I tried to build these bigger graphs makimg sur that there are at least 2 triples with nodes having the same types as target in order to enable the negative sampling, but the AUPRC/AUROC part is still not working on these.

I copy you the data and configuration files, as well as the run command:
python script/run.py -s 1024 -c config/expe_run.yaml --gpus null
brg.txt

entity_names.txt

entity_types.txt

graph_learning.txt

graph_test.txt

graph_validation.txt

expe_run.yaml

The stack error I get is the same one:

/opt/miniconda3/envs/biopathnet/lib/python3.10/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/native/TensorShape.cpp:3484.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
Traceback (most recent call last):
  File "/Users/claudy/work/projects/BioPathNet/script/run.py", line 102, in <module>
    train_and_validate(cfg, solver)
  File "/Users/claudy/work/projects/BioPathNet/script/run.py", line 62, in train_and_validate
    metric = solver.evaluate("valid")
  File "/opt/miniconda3/envs/biopathnet/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/opt/miniconda3/envs/biopathnet/lib/python3.10/site-packages/torchdrug/core/engine.py", line 222, in evaluate
    metric = model.evaluate(pred, target)
  File "/Users/claudy/work/projects/BioPathNet/biopathnet/task.py", line 260, in evaluate
    neg_t = functional.multinomial(prob[:,0,:], 1, replacement=True)
  File "/opt/miniconda3/envs/biopathnet/lib/python3.10/site-packages/torchdrug/layers/functional/functional.py", line 17, in multinomial
    return torch.multinomial(input, num_sample, replacement)
RuntimeError: invalid multinomial distribution (sum of probabilities <= 0)