Examples

Dispel4Py examples

How to test

Run the following command:

python -m dispel4py.simple_process dispel4py.examples.graph_testing.pipeline_test -i 2

The output should look something like this:

Processing 2 iteration(s)
Starting simple processing.
Inputs: {'TestProducer0': [{}, {}]}
Results: {('TestOneInOneOut5', 'output'): [1, 2]}

The above example runs a simple pipeline with 5 connected processing elements and one producer. The number of iterations -i determines how many iterations the producer executes.

A simple example graph

Create a dispel4py PE that produces words selected at random from a list:

from dispel4py.core import GenericPE, TYPE
class RandomWordProducer(GenericPE):
    
    words = ['Lorem', 'ipsum', 'dolor', 'sit', 'amet', 'consectetur', 'adipisicing', 'elit']

    def __init__(self):
        GenericPE.__init__(self)
        self._add_output('output')
        
    def _process(self, inputs):
        outputs = {}
        outputs["output"] = random.choice(RandomWordProducer.words)
        return outputs

Another PE filters the input blocks and decides randomly whether to output a block or not.

from dispel4py.core import GenericPE, TYPE
import random

class RandomFilter(GenericPE):
    
    input_name = 'input'
    output_name = 'output'
    
    def __init__(self):
        GenericPE.__init__(self)
        self._add_input(RandomFilter.input_name)
        self._add_output(RandomFilter.output_name)
    
    def setInputTypes(self, types):
        inputType = types[RandomFilter.input_name]
        self.inputconnections[RandomFilter.input_name][TYPE] = inputType
        self.outputconnections[RandomFilter.output_name][TYPE] = inputType
        
    def _process(self, inputs):
        if random.choice([True, False]):
            return { 'output' : inputs['input'] }

Now create a graph that connects the random word producer to the random filter.

from dispel4py.workflow_graph import WorkflowGraph

words = RandomWordProducer()
filter = RandomFilter()
graph = WorkflowGraph()
graph.connect(words, 'output', filter, 'input')

Store the code above in a module called filter_graph.py.

Execute 10 iterations of the graph with this command:

python -m dispel4py.simple_process filter_graph -i 10

and it should produce output similar to this (since the filter is random you might see a different list of words):

Processing 10 iteration(s)
Starting simple processing.
Inputs: {'RandomWordProducer0': [{}, {}, {}, {}, {}, {}, {}, {}, {}, {}]}
Results: {('RandomFilter1', 'output'): ['consectetur', 'amet', 'ipsum', 'elit', 'dolor']}