A shuffle for sending a key-value pair to multiple processes

[stakizawa]

An usual shuffle operation in MapReduce sends a key-value to just one process. However in some applications, such as the observation operator in weather forecasting by data assimilation, a data (a key-value) generated by a process needs to be sent to multiple processes. If we support such an operation, a variant of shuffle that sends a key-value to multiple processes that satisfy some conditions is required.

Though the same operation can be performed by kmr users by generating multiple key-values whose values are same but keys are different, it consumes many memories.