[SOLVED] Spark Practice with Spark: Dataset Analysis Overview and Rationale

Practice with Spark: Dataset Analysis Overview and Rationale
Sparks intended use is for data lakes which were discussed previously. It is important to be able to process these large data sets effectively with Spark. This assignment will provide you with experience and practice in using Spark to analyze a large data set.
This assignment is directly linked to the following key learning outcomes from the course syllabus:
Course Outcomes
Use methodologies to analyze big data sets and determine insights, including the use of Spark;
For this assignment, you will download and process, with Spark, two of the following datasets OR two datasets your team selected for the final project. Note however, this is an individual assignment. If you choose your project dataset, your group may choose to incorporate your work into their assignment, but that is not required.
[XLSX] [XLSX] [CSV]
Assignment Summary
Dataset File Format
(with download link)
Annual House Price Indexes
(see Working Papers 16-01, 16-02, and 16-04)
-Five-Digit ZIP Codes (Developmental Index; Not Seasonally Adjusted)
Annual House Price Indexes
(see Working Papers 16-01, 16-02, and 16-04)
Three-Digit ZIP Codes (Developmental Index; Not Seasonally Adjusted)
Median Home Value Zillow Home Value Index (ZHVI) by Zip Code
Write a 3-5 report that includes a section for each data set you choose to analyze. For each data set include
A description of the steps you took to perform the analysis, with screen shots

Results of your analysis
Your insights based on your analysis
The paper should follow the following format: (i) Introduction
Format & Guidelines
Provide a short description of the dataset you analyzed and purpose for the analysis. Identify questions you are attempting to answer with or insights you want to gain from the analysis
(ii) Analysis and results
Outline your steps, with screen shots, and provide the results of your analysis. Connect the results and your analysis to the purpose described in the introduction. Be specific.
(iii) Insights
Provide your insights based on your analysis. Connect your insights to the purpose of the analysis.

Spark Practice Rubric
Provides a short Provides a short Provides a description of Clearly outlines steps, Outlines the steps using Provides a general outline
Succinctly provides Provides insights from the Provides insights from the There are no noticeable There are very few errors There are more than five
Category
Meets Standards
Approaching Standards
Below Standards
Introduction
description of the dataset and purpose for the analysis by identifying specific insights to be gained.
description of the dataset and purpose for the analysis, but insights to be gained are vague.
the dataset and purpose of the analysis, but is very vague. Does not identify insights to be gained.
Analysis and Results
with screen shots, and the results of the analysis. Connections between purpose and the analysis are evident and specific.
screen shots, of the analysis and provides results. Connections between the purpose and the analysis are clear.
of steps of the analysis, with few screen shots. Provides the results. Connections between the purpose and the analysis are vague.
Insights
insights from the analysis that are directly connected to the purpose identified in the introduction.
analysis that are generally connected to the purpose identified in the introduction.
analysis, but insights are not connected to the purpose identified in the introduction
Writing Mechanics
errors in grammar, spelling, and punctuation; and completely correct usage of title page, citations, and references.
in grammar, spelling, and punctuation; and completely correct usage of title page, citations, and references.
errors in grammar, spelling, and punctuation; or the usage of title page, citations, and references are incomplete.