Hail via Livy on PIC-SURE#79
Open
DeniseKersjes wants to merge 27 commits intohms-dbmi:masterfrom
Open
Conversation
Duplicate of HAIL.java, but with TODO's in it to connect Hail with IRCT using Livy.
… (gene, significance, subjectID)
Previous multiple tabs in a row where seen as one tab. This issue was solved by converting the output to JSON instead of a string, and rewrite the parser.
This reverts commit 8ef3f2c.
This reverts commit 8905157.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi,
The Hyve (https://thehyve.nl) is putting effort to make cohort selection on whole genome sequencing data in Hail (https://hail.is) possible in Glowing Bear (https://glowingbear.app/). IRCT is used to support the Hail framework. This pull request includes the support for querying Hail via the REST server Livy (https://livy.incubator.apache.org/). Gabor Korodi suggested us to do this implementation instead of the current custom flask-restful server HailProxy (https://github.com/hms-dbmi/HailProxy).
The implementation makes use of the Docker image for Livy over Hail (https://github.com/thehyve/livy-hail-docker). Our docker compose file defines port "8998:8998" and volumes ‘./hail/data:/app/data’. Furthermore, PostgreSQL statements are used to load the resources. I notice there is also the ‘PIC-SURE-resources’ repository (https://github.com/hms-dbmi/PIC-SURE-resources) that contains SQL statements. I created a pull request to this repository too (hms-dbmi/PIC-SURE-resources#8).
Lastly, a note that the implementation at the moment only can handle TSV or MAF files having the column names ‘ID’, ‘GENE’, and ‘CLIN_SIG’.
Best wishes,
Denise Kersjes