MPI-based Creation and Benchmarking of a Dynamic Elasticsearch Cluster

Related publication: Secure Elasticsearch Clusters on HPC Systems for Sensitive Data

READ THE REPORT OR THE PRESENTATION SLIDES

Structure

• benchmarker contains
  • The benchmarking code for ingestion
  • The querying benchmarker
• containers contains
  • A workflow for building a singularity container with elasticsearch bindmounted into it
  • A startup script to build up an elasticsearch cluster
• mrirally contains the code for generating fake data
• latex.tgz contains the latex code for report, presentation as well as the analysis scripts with the benchmark data

What has to be done beforehand!

• Download nyc_taxis documents.json (70GB+) and put it into ./benchmarker/data/nyc_taxis