TypeError: cannot pickle 'weakref' object related to save_session

[Yizhi-Zhang]

Hi, @ludvb

Thank you so much for providing such a fantastic tool. I've encountered some issues while running the software. I came across the error when I ran xfuse run config_demo.toml

Traceback (most recent call last):
  File "/root/.local/lib/python3.8/site-packages/xfuse/run.py", line 145, in _panic
    save_session("exception")
  File "/root/.local/lib/python3.8/site-packages/xfuse/session/io.py", line 33, in save_session
    **{
  File "/root/.local/lib/python3.8/site-packages/xfuse/session/io.py", line 38, in <dictcomp>
    if _can_pickle(k, v)
  File "/root/.local/lib/python3.8/site-packages/xfuse/session/io.py", line 23, in _can_pickle
    _ = pickle.dumps(x)
TypeError: cannot pickle 'weakref' object

It seems like there was an issue while saving the session. Here is my config_demo.toml file.

[xfuse]
# This section defines modeling options. It can usually be left as-is.
network_depth = 6
network_width = 12
gene_regex = "^(?!RPS|RPL|MT-).*" # Regex matching genes to include in the model. By default, exclude mitochondrial and ribosomal genes.
min_counts = 1 # Exclude all genes with fewer reads than this value.

[settings]
cache_data = true
data_workers = 0 # Number of worker processes for data loading. If set to zero, run data loading in main thread.

[expansion_strategy]
# This section contains configuration options for the metagene expansion strategy.
type = "DropAndSplit" # Available choices: Extra, DropAndSplit
purge_interval = 2 # Metagene purging interval (epochs)

[expansion_strategy.Extra]
num_metagenes = 4
anneal_to = 1
anneal_epochs = 1000

[expansion_strategy.DropAndSplit]
max_metagenes = 50

[optimization]
# This section defines options used during training. It may be necessary to decrease the batch or patch size if running out of memory during training.
batch_size = 2
epochs = 10
learning_rate = 0.0003
patch_size = 768 # Size of training patches. Set to '-1' to use as large patches as possible.

[analyses]
# This section defines which analyses to run. Each analysis has its own subtable with configuration options. Remove the table to stop the analysis from being run.

[analyses.analysis-gene_maps]
# Constructs a map of imputed expression for each gene in the dataset.
type = "gene_maps"

[analyses.analysis-gene_maps.options]
gene_regex = ".*"
num_samples = 10
genes_per_batch = 10
predict_mean = true
normalize = false
mask_tissue = true
scale = 1.0
writer = "tensor"


[slides]
# This section defines the slides to use in the experiment. Covariates are specified in the "covariates" table. Slide-specific options can be specified in the "options" table.

[slides.section0]
data = "data/demo/data.h5"

[slides.section0.covariates]
section = "0"

[slides.section0.options]
min_counts = 1
always_filter = []
always_keep = [1]

Could you please help with the issue? Thank you very much.

Yizhi

[ludvb]

Hi Yizhi, Thanks for reporting! I have not seen this before. Can I ask which Python and pytorch versions you are using? I would maybe first try to install the torch version listed in the pyproject.toml file in the root of this repository and check whether the error still occurs:

pip install -U --force-reinstall torch==1.8.1 torchvision==0.9.1

Otherwise, a quick fix would be to catch also TypeErrors in the except clause; i.e., modifying line 24 of xfuse/session/io.py to

except (pickle.PickleError, TypeError) as exc:

[Yizhi-Zhang]

Thank you very much for your assistance. I have resolved the issue, and XFuse is now running smoothly.