Having an issue while building

[d-henness]

I'm having a problem building program. I'm using the environment provided by the env.yml file. Here are the last few lines of log output.

+ rm -rf /var/lib/apt/lists/auxfiles /var/lib/apt/lists/deb.debian.org_debian-security_dists_buster_updates_InRelease /var/lib/apt/lists/deb.debian.org_debian-security_dists_buster_updates_main_binary-amd64_Packages.lz4 /var/lib/apt/lists/deb.debian.org_debian_dists_buster-updates_InRelease /var/lib/apt/lists/deb.debian.org_debian_dists_buster-updates_main_binary-amd64_Packages.lz4 /var/lib/apt/lists/deb.debian.org_debian_dists_buster_InRelease /var/lib/apt/lists/deb.debian.org_debian_dists_buster_main_binary-amd64_Packages.lz4 /var/lib/apt/lists/lock /var/lib/apt/lists/partial
+ echo library(devtools)
+ R --no-save

R version 3.5.2 (2018-12-20) -- "Eggshell Igloo"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(devtools)
>
+ R CMD INSTALL /dndscv/dndscv.tar.gz
Error in untar2(tarfile, files, list, exdir, restore_times) :
  incomplete block on file
FATAL:   While performing build: while running engine: exit status 1
make: *** [containers/dndscv/dndscv.mk:12: ../containers/dndscv.simg] Error 255

Is the dndscv.tar.gz file it mentions supposed to be the one found here https://github.com/im3sanger/dndscv/releases and if so can I replace the file at intogen-plus/build/containers/dndscv/dndscv.tar.gz with that file and have this work?

[d-henness]

I tried the above and got past that step but now I have the following error message.

Building dNdSCV reference
echo "library(dndscv); buildref(\"../datasets/regions/cds_biomart.tsv\", \"../datasets/regions/Homo_sapiens.GRCh38.fa\", outfile = \"../datasets/dndscv/RefCDS.rda\")" | \
        singularity exec ../containers/dndscv.simg R --no-save

R version 3.5.2 (2018-12-20) -- "Eggshell Igloo"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(dndscv); buildref("../datasets/regions/cds_biomart.tsv", "../datasets/regions/Homo_sapiens.GRCh38.fa", outfile = "../datasets/dndscv/RefCDS.rda")
Warning message:
replacing previous import ‘Biostrings::translate’ by ‘seqinr::translate’ when loading ‘dndscv’
Error in buildref("../datasets/regions/cds_biomart.tsv", "../datasets/regions/Homo_sapiens.GRCh38.fa",  :
  could not find function "buildref"
Execution halted
make: *** [datasets/dndscv.mk:12: ../datasets/dndscv/RefCDS.rda] Error 1

I'm not really sure why, according to this https://rdrr.io/github/im3sanger/dndscv/man/buildref.html that function should be in there.

[FedericaBrando]

Hi @d-henness ,

intogen implementation of dndscv is fine-tuned to work in IntOGen pipeline setting. Therefore I am afraid that replacing the tar.gz would not work.

It is important to note that dndscv.tar.gz is agit lfs file. Have you initialized the repo accordingly?

[d-henness]

The steps I did before I altered the dndscv.tar.gz file were

git clone https://github.com/bbglab/intogen-plus.git
cd intogen-plus/build
conda env create -f env.yml
conda activate intogenbuild
export COSMIC_KEY=$(echo "my_email:my_password" | base64)
mv ~/MutPanning.jar containers/mutpanning/
make

Edit: I have installed Git LFS and then cloned the repo again and the build seems to be working so far, I'll report back if that fixes everything. Was the Git LFS dependency mentioned somewhere and I just missed it? If not, it probably should be no?

[d-henness]

I'm now getting this error

Download CGC
python datasets/cgc/download.py --download ../datasets/cgc
Traceback (most recent call last):
  File "datasets/cgc/download.py", line 42, in <module>
    cmdline()
  File "/home/dylan/miniconda3/envs/intogenbuild/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/dylan/miniconda3/envs/intogenbuild/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/dylan/miniconda3/envs/intogenbuild/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/dylan/miniconda3/envs/intogenbuild/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "datasets/cgc/download.py", line 32, in cmdline
    url = r.json()['url']
  File "/home/dylan/miniconda3/envs/intogenbuild/lib/python3.6/site-packages/requests/models.py", line 897, in json
    return complexjson.loads(self.text, **kwargs)
  File "/home/dylan/miniconda3/envs/intogenbuild/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/home/dylan/miniconda3/envs/intogenbuild/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/dylan/miniconda3/envs/intogenbuild/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
make: *** [datasets/cgc/cgc.mk:14: ../datasets/cgc/cancer_gene_census.csv] Error 1

I have the COSMIC_KEY var set, is there something else that needs to be done?

[FedericaBrando]

Hi @d-henness ,

Thanks for reaching out, I've been able to reproduce and debug this error. The problem it seems that COSMIC has changed the way the cancer_gene_census file can be downloaded.

Please follow their steps on their guide: https://cancer.sanger.ac.uk/cosmic/download/cosmic/v102/cancergenecensus

and then you can place the file inside the datasets/cgc/ folder.

We are actively working to improve the dataset building step in order to make intogen-plus run as smoother as possible. Thanks a lot!

[d-henness]

Do I want the GRCh37 or the GRCh38. I noticed that the datasets/cgc/download.py file only tries to get the 37 version unless I'm mistaken.

[FedericaBrando]

Hi @d-henness , thanks for your patience.

I have pushed 2 commits in the develop branch that should fix the error. It is still under testing but in principle it should work. could you checkout that branch and pull those commit and run make datasets cores=[YOUR_N_CORES] .

please let me know if you encounter any error