Annotation: add manually the absplice scores

[PaulineTL]

Hi!

From a previous issues, I was recommended to add manually the absplice scores :

if that does not work, one thing you could do is set ìnclude_absplice: false in the config file, run the annotation pipeline and then add the absplice scores manually later. (you can download prescored absplice scores here, you would have to aggregate them over tissues (max over tisues)

Originally posted by @Marcel-Mueck in #161

I have downloaded the file AbSplice_DNA_hg38_snvs.zip. But then, I am not sure to know what are the following steps, could you detail them?

Thank you in advance for your help!

Pauline

[Marcel-Mueck]

Hey Pauline,
we are currently working on a solution to add absplice scores manually. I will send you the script as soon as I tested it and it will be added to the repository in the near future. Hope this helps,
Regards
Marcel

[Marcel-Mueck]

linking #170

[Marcel-Mueck]

In this PR I added two functions to annotations.py: aggregate_and_concat_absplice and merge_absplice_scores.
How to use:
After downloading and unpacking the precomputed absplice scores from zenodo, and running the annotation pipeline without absplice, run deeprvat_annotations aggregate_and_concat_absplice(inside the deeprvat_annotations environment) with --absplice-dir pointing to the unpacked directory and --ab-splice-agg-score-file pointing to the path where the intermediate aggreagted absplice scores should be saved.
Run deeprvat_annotations merge_absplice_scores afterwards with the annotations, variants, protein coding genes as well as the aggregated absplice scores as input. Finally state where you want the output annotation file stored (--output).

[PaulineTL]

Thank you very much!

However, when I run the following command line:
deeprvat_annotations aggregate-and-concat-absplice --absplice-dir /work/gr-fe/databases/absplice_score/AbSplice_DNA_hg38_snvs --ab-splice-agg-score-file /work/gr-fe/ptheimer/deepRvat/deeprvat/deeprvat_run/output_dir/annotations/absplice_scores_intermediate

I am getting the error: RecursionError: maximum recursion depth exceeded while calling a Python object

I have tried to increase the recursion limit (to 10000) but it does not change anything.

The full error:

/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/dask/dataframe/utils.py:369: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
  _numeric_index_types = (pd.Int64Index, pd.Float64Index, pd.UInt64Index)
/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/dask/dataframe/utils.py:369: FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
  _numeric_index_types = (pd.Int64Index, pd.Float64Index, pd.UInt64Index)
/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/dask/dataframe/utils.py:369: FutureWarning: pandas.UInt64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
  _numeric_index_types = (pd.Int64Index, pd.Float64Index, pd.UInt64Index)
[2025-04-29 15:50:09,444] INFO:deeprvat.annotations.annotations: Reading file ENSG00000064995.tsv.gz
Traceback (most recent call last):
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/bin/deeprvat_annotations", line 33, in <module>
    sys.exit(load_entry_point('deeprvat', 'console_scripts', 'deeprvat_annotations')())
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/work/gr-fe/ptheimer/deepRvat/deeprvat/deeprvat/annotations/annotations.py", line 2116, in aggregate_and_concat_absplice
    ab_splice_res["AbSplice_DNA"] = np.max(
  File "<__array_function__ internals>", line 5, in amax
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/numpy/core/fromnumeric.py", line 2754, in amax
    return _wrapreduction(a, np.maximum, 'max', axis, None, out,
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/numpy/core/fromnumeric.py", line 84, in _wrapreduction
    return reduction(axis=axis, out=out, **passkwargs)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/generic.py", line 11941, in max
    return NDFrame.max(self, axis, skipna, level, numeric_only, **kwargs)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/generic.py", line 11383, in max
    return self._stat_function(
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/generic.py", line 11353, in _stat_function
    return self._reduce(
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/frame.py", line 10879, in _reduce
    result = func(values)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/frame.py", line 10817, in func
    return op(values, axis=axis, skipna=skipna, **kwds)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/nanops.py", line 150, in f
    if _has_infs(result):
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/nanops.py", line 189, in _has_infs
    return np.isinf(result).any()
RecursionError: maximum recursion depth exceeded while calling a Python object

What do you think I can do?

Thank you in advance,
Pauline

[Marcel-Mueck]

Hey Pauline,
I had a look and saw a possible reason for this (a for loop in line 2116 inside np.max is called each row). I modified the script and pushed the changes. Hope this will help to get rid of the recursion error.

[PaulineTL]

I have changed my file annotations.py with your uptades, but sadly I still get the same error:

Traceback (most recent call last):
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/bin/deeprvat_annotations", line 33, in <module>
    sys.exit(load_entry_point('deeprvat', 'console_scripts', 'deeprvat_annotations')())
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/work/gr-fe/ptheimer/deepRvat/deeprvat/deeprvat/annotations/annotations.py", line 2117, in aggregate_and_concat_absplice
    ab_splice_res["AbSplice_DNA"] = np.max(
  File "<__array_function__ internals>", line 5, in amax
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/numpy/core/fromnumeric.py", line 2754, in amax
    return _wrapreduction(a, np.maximum, 'max', axis, None, out,
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/numpy/core/fromnumeric.py", line 84, in _wrapreduction
    return reduction(axis=axis, out=out, **passkwargs)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/generic.py", line 11941, in max
    return NDFrame.max(self, axis, skipna, level, numeric_only, **kwargs)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/generic.py", line 11383, in max
    return self._stat_function(
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/generic.py", line 11353, in _stat_function
    return self._reduce(
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/frame.py", line 10879, in _reduce
    result = func(values)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/frame.py", line 10817, in func
    return op(values, axis=axis, skipna=skipna, **kwds)
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/nanops.py", line 150, in f
    if _has_infs(result):
  File "/home/ptheimer/.local/share/mamba/envs/deeprvat_annotations/lib/python3.9/site-packages/pandas/core/nanops.py", line 189, in _has_infs
    return np.isinf(result).any()
RecursionError: maximum recursion depth exceeded while calling a Python object

Do you think it is due to my memory or disk? But I am running on a cluster.

[Marcel-Mueck]

hey pauline,
I couldn't yet reproduce the recursion error on my end, which made it difficult to tackle this bug. However, I made 2 minor changes to the line 2118: absplice_columns = [j for j in ab_splice_res.columns if "AbSplice_DNA_" in j] which might help to avoid recursion.