-
Notifications
You must be signed in to change notification settings - Fork 1
Dockerise and add a csv writer #13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…f source information
… assume that if we get that far we've written the data
The fix linked is to do with the build cache — I don't think this will affect what the container does at runtime. If you run it as the GID for "docker" on the GAE it should create files owned by root:docker, which should be readable as long as the root mount dir has been created outside of docker. Ideally we'd also not run as root, but I'm looking into getting a service account set up for that. Anyway, this can be a separate issue. (opened #14) |
jeremyestein
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fine, with a few suggestions. Don't mind if they're fixed here or in a separate PR. If we move to parquet, some are not relevant anyway.
…lems when using setup tools to install
Co-authored-by: Jeremy Stein <j.stein@ucl.ac.uk>
A joint pull request addressing #1 and #5. Got slightly jumbled because
jeremy/dockerwas branched off1-csv-writerrather thandevbut OK I think.Creates a docker container for the waveform-controller and a volume to save the data (similar to the saving of raw hl7 this will result in waveform_data directory belonging to root) We can fix this I think by mounting with a difference username (similar to https://github.com/inform-us/INFORMus/issues/1507#issuecomment-2447767038).
Writes csv data to a new directory caller waveform_data. I file per day per source, currently it looks like about 64Mb per day. Maybe OK in the short term but will need addressing soon.