404 not found error in dataman_2019.Rmd

[gillsignals]

Lines 875-877 of dataman_2019.Rmd generate a 404 not found error after authenticating with Google BigQuery and returning to R as directed by the browser:

tcgaCon %>% tbl("Somatic_Mutation") %>% dplyr::filter(project_short_name=="TCGA-GBM") %>% 
       dplyr::select(Variant_Classification, Hugo_Symbol) %>% group_by(Variant_Classification) %>%
       summarise(n=n())
Error: HTTP error [404] Not Found

Is this the appropriate workflow? If so, what do learners need to know or do in order to not encounter this 404 error?

> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 17763)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
 [1] grid      stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] magrittr_1.5                                       dplyr_0.8.3                                       
 [3] bigrquery_1.2.0                                    RaggedExperiment_1.8.0                            
 [5] curatedTCGAData_1.6.0                              MultiAssayExperiment_1.10.4                       
 [7] VariantTools_1.26.0                                VariantAnnotation_1.30.1                          
 [9] ph525x_0.0.48                                      png_0.1-7                                         
[11] ldblock_1.14.2                                     erma_1.0.0                                        
[13] Homo.sapiens_1.3.1                                 TxDb.Hsapiens.UCSC.hg19.knownGene_3.2.2           
[15] OrganismDbi_1.26.0                                 GenomicFeatures_1.36.4                            
[17] GenomicAlignments_1.20.1                           GenomicFiles_1.20.0                               
[19] rtracklayer_1.44.2                                 Rsamtools_2.0.0                                   
[21] RNAseqData.HNRNPC.bam.chr14_0.22.0                 IlluminaHumanMethylation450kanno.ilmn12.hg19_0.6.0
[23] IlluminaHumanMethylation450kmanifest_0.4.0         minfi_1.30.0                                      
[25] bumphunter_1.26.0                                  locfit_1.5-9.1                                    
[27] iterators_1.0.12                                   foreach_1.4.7                                     
[29] Biostrings_2.52.0                                  XVector_0.24.0                                    
[31] data.table_1.12.2                                  GO.db_3.8.2                                       
[33] org.Hs.eg.db_3.8.2                                 airway_1.4.0                                      
[35] SummarizedExperiment_1.14.1                        DelayedArray_0.10.0                               
[37] BiocParallel_1.18.1                                matrixStats_0.54.0                                
[39] GenomicRanges_1.36.0                               GenomeInfoDb_1.20.0                               
[41] ArrayExpress_1.44.0                                GEOquery_2.52.0                                   
[43] annotate_1.62.0                                    XML_3.98-1.20                                     
[45] AnnotationDbi_1.46.1                               IRanges_2.18.1                                    
[47] S4Vectors_0.22.0                                   Biobase_2.44.0                                    
[49] BiocGenerics_0.30.0                                GSE5859Subset_1.0                                 

loaded via a namespace (and not attached):
  [1] tidyselect_0.2.5              RSQLite_2.1.2                 munsell_0.5.0                 codetools_0.2-16             
  [5] preprocessCore_1.46.0         withr_2.1.2                   colorspace_1.4-1              knitr_1.24                   
  [9] rstudioapi_0.10               labeling_0.3                  GenomeInfoDbData_1.2.1        bit64_0.9-7                  
 [13] rhdf5_2.28.0                  vctrs_0.2.0                   xfun_0.9                      BiocFileCache_1.8.0          
 [17] affxparser_1.56.0             R6_2.4.0                      illuminaio_0.26.0             AnnotationFilter_1.8.0       
 [21] bitops_1.0-6                  reshape_0.8.8                 assertthat_0.2.1              promises_1.0.1               
 [25] scales_1.0.0                  gtable_0.3.0                  ensembldb_2.8.0               rlang_0.4.0                  
 [29] zeallot_0.1.0                 genefilter_1.66.0             splines_3.6.1                 lazyeval_0.2.2               
 [33] gargle_0.3.1                  BiocManager_1.30.4            yaml_2.2.0                    reshape2_1.4.3               
 [37] snpStats_1.34.0               backports_1.1.4               httpuv_1.5.1                  RBGL_1.60.0                  
 [41] tools_3.6.1                   nor1mix_1.3-0                 ggplot2_3.2.1                 affyio_1.54.0                
 [45] ff_2.2-14                     RColorBrewer_1.1-2            siggenes_1.58.0               Rcpp_1.0.1                   
 [49] plyr_1.8.4                    progress_1.2.2                zlibbioc_1.30.0               purrr_0.3.2                  
 [53] RCurl_1.95-4.12               prettyunits_1.0.2             openssl_1.4.1                 fs_1.3.1                     
 [57] ProtGenerics_1.16.0           hms_0.5.1                     mime_0.7                      xtable_1.8-4                 
 [61] mclust_5.4.5                  gridExtra_2.3                 compiler_3.6.1                biomaRt_2.40.4               
 [65] tibble_2.1.3                  crayon_1.3.4                  htmltools_0.3.6               later_0.8.0                  
 [69] snow_0.4-3                    tidyr_0.8.3                   oligo_1.48.0                  DBI_1.0.0                    
 [73] ExperimentHub_1.10.0          dbplyr_1.4.2                  MASS_7.3-51.4                 rappdirs_0.3.1               
 [77] EnsDb.Hsapiens.v75_2.99.0     Matrix_1.2-17                 readr_1.3.1                   quadprog_1.5-7               
 [81] pkgconfig_2.0.2               registry_0.5-1                xml2_1.2.2                    rngtools_1.4                 
 [85] pkgmaker_0.27                 multtest_2.40.0               beanplot_1.2                  bibtex_0.4.2                 
 [89] doRNG_1.7.1                   scrime_1.3.5                  stringr_1.4.0                 digest_0.6.20                
 [93] graph_1.62.0                  base64_2.0                    DelayedMatrixStats_1.6.0      curl_4.0                     
 [97] shiny_1.3.2                   jsonlite_1.6                  nlme_3.1-141                  Rhdf5lib_1.6.0               
[101] askpass_1.1                   limma_3.40.6                  BSgenome_1.52.0               pillar_1.4.2                 
[105] lattice_0.20-38               httr_1.4.1                    survival_2.44-1.1             interactiveDisplayBase_1.22.0
[109] glue_1.3.1                    UpSetR_1.4.0                  bit_1.1-14                    stringi_1.4.3                
[113] HDF5Array_1.12.2              blob_1.2.0                    oligoClasses_1.46.0           AnnotationHub_2.16.1         
[117] memoise_1.1.0     

Thanks!

[vjcitn]

Did this part succeed?

dbListTables(tcgaCon)

The bigrquery package is requesting access to your Google account. Select a pre-authorised account or enter '0' to obtain a new token. Press Esc/Ctrl + C to abort. 1: stvjc@channing.harvard.edu Selection: 0 *Waiting for authentication in browser...* *Press Esc/Ctrl + C to abort* *Authentication complete.* [1] "Copy_Number_Segment_Masked" "Copy_Number_Segment_Masked_r14" [3] "DNA_Methylation" "DNA_Methylation_chr1" [5] "DNA_Methylation_chr10" "DNA_Methylation_chr11" [7] "DNA_Methylation_chr12" "DNA_Methylation_chr13" ... The command you mention does work if the tcgaCon connection is valid. CGC_BILLING needs to be an environment variable that has been set before the R session started, to refer to an active google cloud platform project id.

…

On Mon, Sep 9, 2019 at 9:45 PM gillsignals ***@***.***> wrote: Lines 875-877 of dataman_2019.Rmd generate a 404 not found error after authenticating with Google BigQuery and returning to R as directed by the browser: tcgaCon %>% tbl("Somatic_Mutation") %>% dplyr::filter(project_short_name=="TCGA-GBM") %>% dplyr::select(Variant_Classification, Hugo_Symbol) %>% group_by(Variant_Classification) %>% summarise(n=n()) Error: HTTP error [404] Not Found Is this the appropriate workflow? If so, what do learners need to know or do in order to not encounter this 404 error? > sessionInfo() R version 3.6.1 (2019-07-05) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 17763) Matrix products: default locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C LC_TIME=English_United States.1252 attached base packages: [1] grid stats4 parallel stats graphics grDevices utils datasets methods base other attached packages: [1] magrittr_1.5 dplyr_0.8.3 [3] bigrquery_1.2.0 RaggedExperiment_1.8.0 [5] curatedTCGAData_1.6.0 MultiAssayExperiment_1.10.4 [7] VariantTools_1.26.0 VariantAnnotation_1.30.1 [9] ph525x_0.0.48 png_0.1-7 [11] ldblock_1.14.2 erma_1.0.0 [13] Homo.sapiens_1.3.1 TxDb.Hsapiens.UCSC.hg19.knownGene_3.2.2 [15] OrganismDbi_1.26.0 GenomicFeatures_1.36.4 [17] GenomicAlignments_1.20.1 GenomicFiles_1.20.0 [19] rtracklayer_1.44.2 Rsamtools_2.0.0 [21] RNAseqData.HNRNPC.bam.chr14_0.22.0 IlluminaHumanMethylation450kanno.ilmn12.hg19_0.6.0 [23] IlluminaHumanMethylation450kmanifest_0.4.0 minfi_1.30.0 [25] bumphunter_1.26.0 locfit_1.5-9.1 [27] iterators_1.0.12 foreach_1.4.7 [29] Biostrings_2.52.0 XVector_0.24.0 [31] data.table_1.12.2 GO.db_3.8.2 [33] org.Hs.eg.db_3.8.2 airway_1.4.0 [35] SummarizedExperiment_1.14.1 DelayedArray_0.10.0 [37] BiocParallel_1.18.1 matrixStats_0.54.0 [39] GenomicRanges_1.36.0 GenomeInfoDb_1.20.0 [41] ArrayExpress_1.44.0 GEOquery_2.52.0 [43] annotate_1.62.0 XML_3.98-1.20 [45] AnnotationDbi_1.46.1 IRanges_2.18.1 [47] S4Vectors_0.22.0 Biobase_2.44.0 [49] BiocGenerics_0.30.0 GSE5859Subset_1.0 loaded via a namespace (and not attached): [1] tidyselect_0.2.5 RSQLite_2.1.2 munsell_0.5.0 codetools_0.2-16 [5] preprocessCore_1.46.0 withr_2.1.2 colorspace_1.4-1 knitr_1.24 [9] rstudioapi_0.10 labeling_0.3 GenomeInfoDbData_1.2.1 bit64_0.9-7 [13] rhdf5_2.28.0 vctrs_0.2.0 xfun_0.9 BiocFileCache_1.8.0 [17] affxparser_1.56.0 R6_2.4.0 illuminaio_0.26.0 AnnotationFilter_1.8.0 [21] bitops_1.0-6 reshape_0.8.8 assertthat_0.2.1 promises_1.0.1 [25] scales_1.0.0 gtable_0.3.0 ensembldb_2.8.0 rlang_0.4.0 [29] zeallot_0.1.0 genefilter_1.66.0 splines_3.6.1 lazyeval_0.2.2 [33] gargle_0.3.1 BiocManager_1.30.4 yaml_2.2.0 reshape2_1.4.3 [37] snpStats_1.34.0 backports_1.1.4 httpuv_1.5.1 RBGL_1.60.0 [41] tools_3.6.1 nor1mix_1.3-0 ggplot2_3.2.1 affyio_1.54.0 [45] ff_2.2-14 RColorBrewer_1.1-2 siggenes_1.58.0 Rcpp_1.0.1 [49] plyr_1.8.4 progress_1.2.2 zlibbioc_1.30.0 purrr_0.3.2 [53] RCurl_1.95-4.12 prettyunits_1.0.2 openssl_1.4.1 fs_1.3.1 [57] ProtGenerics_1.16.0 hms_0.5.1 mime_0.7 xtable_1.8-4 [61] mclust_5.4.5 gridExtra_2.3 compiler_3.6.1 biomaRt_2.40.4 [65] tibble_2.1.3 crayon_1.3.4 htmltools_0.3.6 later_0.8.0 [69] snow_0.4-3 tidyr_0.8.3 oligo_1.48.0 DBI_1.0.0 [73] ExperimentHub_1.10.0 dbplyr_1.4.2 MASS_7.3-51.4 rappdirs_0.3.1 [77] EnsDb.Hsapiens.v75_2.99.0 Matrix_1.2-17 readr_1.3.1 quadprog_1.5-7 [81] pkgconfig_2.0.2 registry_0.5-1 xml2_1.2.2 rngtools_1.4 [85] pkgmaker_0.27 multtest_2.40.0 beanplot_1.2 bibtex_0.4.2 [89] doRNG_1.7.1 scrime_1.3.5 stringr_1.4.0 digest_0.6.20 [93] graph_1.62.0 base64_2.0 DelayedMatrixStats_1.6.0 curl_4.0 [97] shiny_1.3.2 jsonlite_1.6 nlme_3.1-141 Rhdf5lib_1.6.0 [101] askpass_1.1 limma_3.40.6 BSgenome_1.52.0 pillar_1.4.2 [105] lattice_0.20-38 httr_1.4.1 survival_2.44-1.1 interactiveDisplayBase_1.22.0 [109] glue_1.3.1 UpSetR_1.4.0 bit_1.1-14 stringi_1.4.3 [113] HDF5Array_1.12.2 blob_1.2.0 oligoClasses_1.46.0 AnnotationHub_2.16.1 [117] memoise_1.1.0 Thanks! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#82?email_source=notifications&email_token=ABDI5QRX2AYXVARKTO4HTR3QI333FA5CNFSM4IVB4AFKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HKKH4DQ>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDI5QSB3ITANZV7WPXDIJLQI333FANCNFSM4IVB4AFA> .

-- The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance  HelpLine at http://www.partners.org/complianceline <http://www.partners.org/complianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

[gillsignals]

The first part did work after authentication in the browser, but the subsequent command yields the 404 error.

Is a valid Google Cloud platform project ID strictly required to perform this operation? I would strongly suppose that 95% of learners in the course will not have this ID and be unable to run this code when they copy it to their personal machine.

We try to have all the code in our courses be fully replicable by learners if they copy-paste or download the Rmd files. Is there any way to do this operation without requiring a project/billing ID or environment variable?

[vjcitn]

On Tue, Sep 10, 2019 at 6:31 AM gillsignals ***@***.***> wrote: The first part did work after authentication in the browser, but the subsequent command yields the 404 error. Is a valid Google Cloud platform project ID strictly required to perform this operation? I would strongly suppose that 95% of learners in the course will not have this ID and be unable to run this code when they copy it to their personal machine. We try to have all the code in our courses be fully replicable by learners if they copy-paste or download the Rmd files. Is there any way to do this operation without requiring a project/billing ID or environment variable?

No. It should be removed or made optional for those who are willing to get a project id. Can you issue a pull request that removes or makes these computational conditional on availability of a non-null value for CGC_BILLING?

…

— You are receiving this because you commented. Reply to this email directly, view it on GitHub <#82?email_source=notifications&email_token=ABDI5QTSIE6IMXFRG4I4OFTQI5ZP5A5CNFSM4IVB4AFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6KUDMY#issuecomment-529875379>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDI5QR4JH6TJOAV4HDUA3LQI5ZP5ANCNFSM4IVB4AFA> .

-- The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance  HelpLine at http://www.partners.org/complianceline <http://www.partners.org/complianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

[gillsignals]

The reason I haven't fixed this yet is because the problem sets still require the results of this operation. When we meet, we can talk about fully replacing those exercises and/or moving this to an optional section without exercises (and possibly in course 6 instead of course 5, since it's more advanced).

Thanks!