Overview

Topics covered:

Map Reduce - Allows for complex problems to be broken down into smaller parts using a master and a worker system. There are two parts to this, the "Map" step and the "Reduce" step. In the Map step the master node breaks up the problem and gives the parts to the worker nodes. The worker nodes can pass this on to other nodes in order to simplify the problem even further. In the Reduce step the partial solutions are all sent back to the master node to be recombined in order to get the output. This method allows for multiple tasks to be done in parallel.
Master Worker - The process used in Map Reduce above, where the master distributes the load among the workers.
REDIS - It is a open source, in memory, key-value data storage. It is very much like a database which does not use SQL and does not have a fixed schema, which means that it is very flexible and the constraints are not as rigid.
Cloud Computing - The main aspect of cloud computing is that there is a virtualization layer that allows multiple OS's to operate at the same time. This leads to multiple people being able to use hardware at the same time and to customize it to their needs(think SaaS, PaaS, IaaS). The consequence for this though is that the speed of your calculations decrease as more people use the same hardware, as well as latency and connection bottlenecks vs GRID.

Readings:

Current Tasks: