Single-Node IOWarp Testing (CS546 Student Project)

[lukemartinlogan]

We want to answer the following questions

1. How well do we compare in terms of performance to similar existing works. Redis is our primary target right now.
2. How well does IOWarp perform compared to traditional filesystem over science workloads?

Place all these pipeline tests under a new folder called jarvis_iowarp/pipelines/single_node

Comparing to Existing Work

Write a jarvis pipeline test script for redis and wrp_cte + wrp_cte_bench.

For both systems, we should vary the following parameters:

1. I/O size [1KB, 4KB, 16KB, 64KB, 128KB, 256KB, 1MB, 16MB]
2. Number of clients: [1, 2, 4, 8, 16]

The benchmark should run for at least 15 seconds for each case.
We will use only DRAM for storage in these cases.

For IOWarp, we should additionally vary the client transport:

1. [shm, ipc, tcp]

To see parameters to this, run these commands:

jarvis pkg help redis
jarvis pkg help wrp_cte
jarvis pkg help wrp_cte_bench

Understanding the Science Workload

Write a pipeline test to experiment with Gray Scott workload. Try to understand how various parameters affect performance and I/O. This is a chemical reaction simulation that models the fluid density of two chemicals in a grid. It's main parameters (to my knowledge) are:

• L is the characteristic dimension of a cube. It will produce an L x L x L grid. This parameter affects memory utilization. If set too high, it will crash. This also affects how much I/O happens at any particular point.
• steps is the number of times we recalculate fluid density. Think of it like the number of frames in a video.
• plotgap is how much we delay before performing I/O. So if we have 10 steps and plotgap 5, it will only do I/O twice.
• nprocs is the number of processes to spawn for the simulation.

On your system, try playing around with this to see how much data gets produced. Goal is to get around 20GB of data in total for the workload. Produce some plots to explain how these things affect I/O size and runtime.

Comparing science workloads

Write a pipeline test to experiment with Gray Scott over iowarp. Vary:

1. The number of processes in Gray Scott [1, 2, 4, 8, 16] (up to 1.5x hardware threads on your system)
2. The number of workers in iowarp: [1, 2, 4, 8, 16] (up to 1.5x hardware threads on your system)