From 22356a5c1e0cd086097275f75e33f7ac8d190dda Mon Sep 17 00:00:00 2001 From: ElMowaled Date: Fri, 7 Feb 2020 15:46:28 +0000 Subject: [PATCH 01/12] First Commit --- Lab2/img/enterprise.png | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 Lab2/img/enterprise.png diff --git a/Lab2/img/enterprise.png b/Lab2/img/enterprise.png new file mode 100644 index 0000000..e69de29 From 00c7c7b1d3976bd46aa47c65400b6345dcef6134 Mon Sep 17 00:00:00 2001 From: ElMowaled Date: Fri, 7 Feb 2020 15:46:43 +0000 Subject: [PATCH 02/12] First Commit --- .idea/encodings.xml | 4 + .idea/modules.xml | 8 ++ .idea/serverless-data-analytics.iml | 9 ++ .idea/vcs.xml | 6 ++ .idea/workspace.xml | 154 ++++++++++++++++++++++++++++ Lab2/.DS_Store | Bin 0 -> 6148 bytes Lab2/README.md | 2 +- Lab2/img/enterprise.png | Bin 0 -> 209877 bytes 8 files changed, 182 insertions(+), 1 deletion(-) create mode 100644 .idea/encodings.xml create mode 100644 .idea/modules.xml create mode 100644 .idea/serverless-data-analytics.iml create mode 100644 .idea/vcs.xml create mode 100644 .idea/workspace.xml create mode 100644 Lab2/.DS_Store diff --git a/.idea/encodings.xml b/.idea/encodings.xml new file mode 100644 index 0000000..15a15b2 --- /dev/null +++ b/.idea/encodings.xml @@ -0,0 +1,4 @@ + + + + \ No newline at end of file diff --git a/.idea/modules.xml b/.idea/modules.xml new file mode 100644 index 0000000..16dc351 --- /dev/null +++ b/.idea/modules.xml @@ -0,0 +1,8 @@ + + + + + + + + \ No newline at end of file diff --git a/.idea/serverless-data-analytics.iml b/.idea/serverless-data-analytics.iml new file mode 100644 index 0000000..d6ebd48 --- /dev/null +++ b/.idea/serverless-data-analytics.iml @@ -0,0 +1,9 @@ + + + + + + + + + \ No newline at end of file diff --git a/.idea/vcs.xml b/.idea/vcs.xml new file mode 100644 index 0000000..35eb1dd --- /dev/null +++ b/.idea/vcs.xml @@ -0,0 +1,6 @@ + + + + + + \ No newline at end of file diff --git a/.idea/workspace.xml b/.idea/workspace.xml new file mode 100644 index 0000000..bb52333 --- /dev/null +++ b/.idea/workspace.xml @@ -0,0 +1,154 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - @@ -119,10 +124,10 @@ - + - + @@ -136,7 +141,7 @@ - + @@ -159,8 +164,8 @@ - - + + diff --git a/Lab2/README.md b/Lab2/README.md index b214709..8bfce5a 100644 --- a/Lab2/README.md +++ b/Lab2/README.md @@ -104,7 +104,7 @@ Now that you have created the table you need to add the partition metadata to th ``` The returned result will contain information for the partitions that are added to NYTaxiRides for each taxi type (yellow, green, fhv) for every month for the year from 2009 to 2016 -## Signing up for Amazon Quicksight Standard Edition +## Signing up for Amazon Quicksight Enterprise Edition 1. Open the [AWS Management Console for QuickSight](https://us-east-1.quicksight.aws.amazon.com/sn/start). From 96b7191db6007dd2a5cbfb29cbec27103406ad34 Mon Sep 17 00:00:00 2001 From: ElMowaled Date: Tue, 28 Apr 2020 15:41:21 +0100 Subject: [PATCH 12/12] Starting from Lab1 --- .idea/workspace.xml | 33 ++++++++++++++++++++++++++++ Lab1/README.md | 10 +++++++++ {Lab2 => Lab1}/img/athena-setup.png | Bin Lab2/README.md | 9 ++++---- 4 files changed, 48 insertions(+), 4 deletions(-) rename {Lab2 => Lab1}/img/athena-setup.png (100%) diff --git a/.idea/workspace.xml b/.idea/workspace.xml index b62b055..6d1a5d3 100644 --- a/.idea/workspace.xml +++ b/.idea/workspace.xml @@ -1,10 +1,14 @@ +<<<<<<< Updated upstream +======= + +>>>>>>> Stashed changes +<<<<<<< Updated upstream +======= + +>>>>>>> Stashed changes @@ -124,10 +137,16 @@ +<<<<<<< Updated upstream +======= + + + +>>>>>>> Stashed changes @@ -141,7 +160,11 @@ +<<<<<<< Updated upstream +======= + +>>>>>>> Stashed changes @@ -171,10 +194,20 @@ +<<<<<<< Updated upstream +======= + + + + + + + +>>>>>>> Stashed changes diff --git a/Lab1/README.md b/Lab1/README.md index 1e777aa..7e4e0da 100644 --- a/Lab1/README.md +++ b/Lab1/README.md @@ -17,6 +17,16 @@ ## Architectural Diagram ![architecture-overview-lab1.png](https://s3.amazonaws.com/us-east-1.data-analytics/labcontent/reinvent2017content-abd313/lab1/Screen+Shot+2017-11-17+at+1.11.18+AM.png) +### Setting up Athena (first time users) + +If you’re a first time Athena user, you might need to configure an S3 bucket, where Athena will store the query results. + +![image](../Lab1/img/athena-setup.png) + +You can use an already existing bucket with a dedicated folder or you can create a new, dedicated bucket. + +NOTE: Make sure you have forward slash at the end of the S3 path + ## Creating Amazon Athena Database and Table Amazon Athena uses Apache Hive to define tables and create databases. Databases are a logical grouping of tables. When you create a database and table in Athena, you are simply describing the schema and location of the table data in Amazon S3\. In case of Hive, databases and tables don’t store the data along with the schema definition unlike traditional relational database systems. The data is read from Amazon S3 only when you query the table. The other benefit of using Hive is that the metastore found in Hive can be used in many other big data applications such as Spark, Hadoop, and Presto. With Athena catalog, you can now have Hive-compatible metastore in the cloud without the need for provisioning a Hadoop cluster or RDS instance. For guidance on databases and tables creation refer [Apache Hive documentation](https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL). The following steps provide guidance specifically for Amazon Athena. diff --git a/Lab2/img/athena-setup.png b/Lab1/img/athena-setup.png similarity index 100% rename from Lab2/img/athena-setup.png rename to Lab1/img/athena-setup.png diff --git a/Lab2/README.md b/Lab2/README.md index 8bfce5a..48c3e30 100644 --- a/Lab2/README.md +++ b/Lab2/README.md @@ -1,9 +1,10 @@ # Lab 2: Visualization using Amazon QuickSight - + * [Signing up for Amazon Quicksight Standard Edition](#signing-up-for-amazon-quicksight-standard-edition) * [Configuring Amazon QuickSight to use Amazon Athena as data source](#configuring-amazon-quicksight-to-use-amazon-athena-as-data-source) * [Visualizing the data using Amazon QuickSight](#visualizing-the-data-using-amazon-quicksight) @@ -17,7 +18,7 @@ ## Architectural Diagram ![architecture-overview-lab2.png](https://s3.amazonaws.com/us-east-1.data-analytics/labcontent/reinvent2017content-abd313/lab2/architecture-overview-lab2.png) - + ## Signing up for Amazon Quicksight Enterprise Edition 1. Open the [AWS Management Console for QuickSight](https://us-east-1.quicksight.aws.amazon.com/sn/start).