From ea0dd1c0b1d216ebaaba004a53b43cbb5304431b Mon Sep 17 00:00:00 2001 From: davidwells Date: Mon, 12 Jun 2017 14:26:18 -0700 Subject: [PATCH] add pywren post --- .../2017-xx-xx-xx-pararrel-lambda-pattern.md | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 posts/2017-xx-xx-xx-pararrel-lambda-pattern.md diff --git a/posts/2017-xx-xx-xx-pararrel-lambda-pattern.md b/posts/2017-xx-xx-xx-pararrel-lambda-pattern.md new file mode 100644 index 000000000..ea8e9c186 --- /dev/null +++ b/posts/2017-xx-xx-xx-pararrel-lambda-pattern.md @@ -0,0 +1,20 @@ +--- +title: Leveraging the full power of Lambda with Parallel execution +description: Lambda can be leveraged to complete massive compute tasks in a fraction of the time it has taken in the past. +date: 2017-06-12 +layout: Post +authors: + - DavidWells +--- + +Post is WIP. Contributions welcome! + +Eric Jonas creator of pywren talks about how they use Lambda for parallel processing data science jobs (one of the most interesting use cases IMO) + +Video: https://youtu.be/OskQytBBdJU?t=13m11s + +Resources + +- https://blog.sungardas.com/CTOLabs/2016/02/aws-lambda-invasion-in-big-data/ +- http://tothestars.io/blog/2016/11/2/serverless-mapreduce +- https://blog.seanssmith.com/posts/pywren-web-scraping.html https://twitter.com/seanwssmith/status/851134826354802694