+ 1581088491183
+
+
+
+
+<<<<<<< Updated upstream
+
+
+
+
+=======
+
+
+
+>>>>>>> Stashed changes
+
+
+
+
+
+
+
+
+
+
+
+
+
+<<<<<<< Updated upstream
+
+=======
+
+>>>>>>> Stashed changes
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+<<<<<<< Updated upstream
+
+
+
+
+=======
+
+
+
+
+
+
+
+>>>>>>> Stashed changes
+
+
+
+
+
\ No newline at end of file
diff --git a/Lab1/README.md b/Lab1/README.md
index 1e777aa..7e4e0da 100644
--- a/Lab1/README.md
+++ b/Lab1/README.md
@@ -17,6 +17,16 @@
## Architectural Diagram

+### Setting up Athena (first time users)
+
+If you’re a first time Athena user, you might need to configure an S3 bucket, where Athena will store the query results.
+
+
+
+You can use an already existing bucket with a dedicated folder or you can create a new, dedicated bucket.
+
+NOTE: Make sure you have forward slash at the end of the S3 path
+
## Creating Amazon Athena Database and Table
Amazon Athena uses Apache Hive to define tables and create databases. Databases are a logical grouping of tables. When you create a database and table in Athena, you are simply describing the schema and location of the table data in Amazon S3\. In case of Hive, databases and tables don’t store the data along with the schema definition unlike traditional relational database systems. The data is read from Amazon S3 only when you query the table. The other benefit of using Hive is that the metastore found in Hive can be used in many other big data applications such as Spark, Hadoop, and Presto. With Athena catalog, you can now have Hive-compatible metastore in the cloud without the need for provisioning a Hadoop cluster or RDS instance. For guidance on databases and tables creation refer [Apache Hive documentation](https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL). The following steps provide guidance specifically for Amazon Athena.
diff --git a/Lab1/img/athena-setup.png b/Lab1/img/athena-setup.png
new file mode 100644
index 0000000..917847d
Binary files /dev/null and b/Lab1/img/athena-setup.png differ
diff --git a/Lab2/.DS_Store b/Lab2/.DS_Store
new file mode 100644
index 0000000..940e942
Binary files /dev/null and b/Lab2/.DS_Store differ
diff --git a/Lab2/README.md b/Lab2/README.md
index 036c572..48c3e30 100644
--- a/Lab2/README.md
+++ b/Lab2/README.md
@@ -1,9 +1,10 @@
# Lab 2: Visualization using Amazon QuickSight
-
+
* [Signing up for Amazon Quicksight Standard Edition](#signing-up-for-amazon-quicksight-standard-edition)
* [Configuring Amazon QuickSight to use Amazon Athena as data source](#configuring-amazon-quicksight-to-use-amazon-athena-as-data-source)
* [Visualizing the data using Amazon QuickSight](#visualizing-the-data-using-amazon-quicksight)
@@ -17,14 +18,14 @@
## Architectural Diagram

-
+
+## Signing up for Amazon Quicksight Enterprise Edition
1. Open the [AWS Management Console for QuickSight](https://us-east-1.quicksight.aws.amazon.com/sn/start).
@@ -117,9 +116,9 @@ The returned result will contain information for the partitions that are added t
> **Note:** Chrome browser might timeout at this step. If that's the case, try this step in Firefox/Microsoft Edge/Safari.
-4. On the next page, for the subscription type select the **"Standard Edition"** and click **Continue**.
+4. On the next page, for the subscription type select the **"Enterprise Edition"** and click **Continue**.
-
+
5. On the next page,
@@ -133,7 +132,7 @@ The returned result will contain information for the partitions that are added t
v. **Click Finish**.
- 
+ 
vi. You will be presented with a message **Congratulations**! **You are signed up for Amazon QuickSight!** on successful sign up. Click on **Go to Amazon QuickSight**.
@@ -141,7 +140,7 @@ The returned result will contain information for the partitions that are added t
Now, on the Amazon QuickSight dashboard, navigate to User Settings page on the Top-Right section and click **Manage QuickSight**.
-
+ 
7. In this section, click on **Security & permissions** and then click **Add or remove**.
@@ -163,28 +162,28 @@ Now, on the Amazon QuickSight dashboard, navigate to User Settings page on the T
> For this lab, you will need to choose the **US West (Oregon)** region.
-
+
1. Click on the region icon on the top-right corner of the page, and select **US West (Oregon)**.
2. Click on **Manage data** on the top-right corner of the webpage to review existing data sets.
-
+
3. Click on **New data set** on the top-left corner of the webpage and review the options.
4. Select **Athena** as a Data source.
-
+
5. Enter the **Data source** **name** (e.g. *AthenaDataSource*).
-
+
6. Click **Create data source**.
7. Select the **mydatabase** database.
-
+
8. Choose the **nytaxirides** table.
9. Choose **Edit/Preview** data.
@@ -249,7 +248,7 @@ Now that you have configured the data source and created a new field to represen
vi. Click **Close**.
- 
+
### Add the month based filter for the month of January
@@ -280,7 +279,7 @@ Now that you have configured the data source and created a new field to represen
vi. Click **Close**.
- 
+
### Visualize the data by hour of day for the month of January 2016
@@ -299,14 +298,33 @@ Now that you have configured the data source and created a new field to represen
3. Select **pickup_datetime** for x-axis by clicking on the **pickup_datetime** field name from **Fields list**.
4. Select **type** for Color by clicking on the **type** field name from **Fields list.**
-5. Click on the field name **pickup_datetime** in x-axis to reveal a sub-menu.
+5. Click on the field name **pickup_datetime** in top **Field Wells** bar to reveal a sub-menu.
6. Select **Aggregate:Day** to aggregate by day.
-
+
8. Using the slider on x-axis, select the entire month of January 2016 for **pickup_datetime** field.
-
+
+
+### Forecast Monthly Number of Pickups Trend
+
+1. Create a new visual by clicking on **Add** in the top left corner and choosing **Add visual**.
+
+
+
+2. Under the **Fields list**, Select **pickup_datetime** for x-axis by clicking on the **pickup_datetime** field name.
+3. Change the visual type to a line chart by selecting the line chart icon highlighted in the screenshot below under **Visual types**.
+4. Click on the field name **pickup_datetime** in top **Field Wells** bar to reveal a sub-menu.
+5. Select **Aggregate:Month** to aggregate by month.
+
+
+
+6. Click arrow in top right corner of the visual and select **Add forecast**.
+
+
+
+
> Note: The interesting outlier in the above graph is that on Jan23rd, 2016, you see the dip in the number of taxis across all types. Doing a quick google search for that date, gets us this weather article from NBC New York
> 
diff --git a/Lab2/img/aggregate-by-day.png b/Lab2/img/aggregate-by-day.png
new file mode 100644
index 0000000..eb8bc79
Binary files /dev/null and b/Lab2/img/aggregate-by-day.png differ
diff --git a/Lab2/img/athena-ds.png b/Lab2/img/athena-ds.png
new file mode 100644
index 0000000..5593dc2
Binary files /dev/null and b/Lab2/img/athena-ds.png differ
diff --git a/Lab2/img/athena-table.png b/Lab2/img/athena-table.png
new file mode 100644
index 0000000..f6040d9
Binary files /dev/null and b/Lab2/img/athena-table.png differ
diff --git a/Lab2/img/connectors.png b/Lab2/img/connectors.png
new file mode 100644
index 0000000..28d8a1e
Binary files /dev/null and b/Lab2/img/connectors.png differ
diff --git a/Lab2/img/create-bucket.png b/Lab2/img/create-bucket.png
new file mode 100644
index 0000000..5f04341
Binary files /dev/null and b/Lab2/img/create-bucket.png differ
diff --git a/Lab2/img/end.png b/Lab2/img/end.png
new file mode 100644
index 0000000..36ce2ea
Binary files /dev/null and b/Lab2/img/end.png differ
diff --git a/Lab2/img/enterprise.png b/Lab2/img/enterprise.png
new file mode 100644
index 0000000..b03b5bf
Binary files /dev/null and b/Lab2/img/enterprise.png differ
diff --git a/Lab2/img/entire-month.png b/Lab2/img/entire-month.png
new file mode 100644
index 0000000..bd9f4ad
Binary files /dev/null and b/Lab2/img/entire-month.png differ
diff --git a/Lab2/img/forecast.png b/Lab2/img/forecast.png
new file mode 100644
index 0000000..9ce211c
Binary files /dev/null and b/Lab2/img/forecast.png differ
diff --git a/Lab2/img/month-filter.png b/Lab2/img/month-filter.png
new file mode 100644
index 0000000..d9e18c1
Binary files /dev/null and b/Lab2/img/month-filter.png differ
diff --git a/Lab2/img/new-account-fields.png b/Lab2/img/new-account-fields.png
new file mode 100644
index 0000000..2ab51b8
Binary files /dev/null and b/Lab2/img/new-account-fields.png differ
diff --git a/Lab2/img/new-visual.png b/Lab2/img/new-visual.png
new file mode 100644
index 0000000..364b405
Binary files /dev/null and b/Lab2/img/new-visual.png differ
diff --git a/Lab2/img/oregon-manage.png b/Lab2/img/oregon-manage.png
new file mode 100644
index 0000000..b92b2c1
Binary files /dev/null and b/Lab2/img/oregon-manage.png differ
diff --git a/Lab2/img/oregon.png b/Lab2/img/oregon.png
new file mode 100644
index 0000000..5df20c0
Binary files /dev/null and b/Lab2/img/oregon.png differ
diff --git a/Lab2/img/prepare-forecast.png b/Lab2/img/prepare-forecast.png
new file mode 100644
index 0000000..fbaffca
Binary files /dev/null and b/Lab2/img/prepare-forecast.png differ
diff --git a/Lab2/img/quicksight-first-page.png b/Lab2/img/quicksight-first-page.png
new file mode 100644
index 0000000..6442b9d
Binary files /dev/null and b/Lab2/img/quicksight-first-page.png differ
diff --git a/Lab2/img/year-filter.png b/Lab2/img/year-filter.png
new file mode 100644
index 0000000..ab4d452
Binary files /dev/null and b/Lab2/img/year-filter.png differ