HW6_Description

1. Objectives

• Get experience with Python programming language and Flask framework.
• Get experience creating web pages using HTML, CSS, JavaScript, DOM, JSON format and XMLHttpRequest object.
• Get experience with Tiingo Stocks API and orgs News API Getting hands-on experience in GCP, AWS or Azure.

1.1 Cloud Exercise

• The backend of this homework must be implemented in the cloud on GCP, AWS or Azure using Python.
• See homework 5 for installation of either one of these platforms. You only have to select one platform to implement your backend.
• See the hints in section 3; a lot of reference material is provided to you.
• For Python and Flask kick-start, please refer to the Lecture slides on the class website.
• You must refer to the grading guidelines, the video, the specs and Piazza. Styling will be graded and the points breakup is mentioned in the grading guidelines.
• The link for the video is https://youtu.be/1vOis6BxYq4

2. Description

In this exercise, you are asked to create a webpage that allows you to search for stock information using the Tiingo Stock API, and the results will be displayed in both tabular format and charts format using HighCharts. You will also provide news articles for the selected stock using the News API.

2.1. Description of the Search Form

The user first opens a web page as shown below in Figure 1, the initial search page, where he/she can enter a stock ticker symbol. Providing a value for the Stock Ticker Symbol field is mandatory.

Figure 1: Initial search page

The search form has two buttons:

1. Search button:

Once the user has provided valid data (a stock ticker is required), and clicked on the Search button, your front-end JavaScript script will make a request to your web server providing it with the form data that was entered (the ticker symbol). You must use GET to transfer the form data to your web server (do not use POST, as you would be unable to provide a sample link to your cloud services). A Python script using Flask will retrieve the data and send it to the Tiingo Stocks API. You need to use the Flask Python framework to make all the API calls.

Using XMLHttpRequest or any other JavaScript calls for anything other than calling your own cloud backend will lead to a 4-point penalty. Do not call the Tiingo Stocks API directly from JavaScript.

Define routing endpoints and make your API call from the Python backend. The recommended tutorial for Flask and more importantly, routing, can be found at the following link: https://flask.palletsprojects.com/en/1.1.x/

If the user clicks on the Search button without providing a value in the field, an alert should pop up Please fill out this field (an example is shown in Figure 2).

Figure 2: An Example of Empty Input alert

2. Clear button:

This button must clear the result area (below the search area) and the text field (stock ticker symbol).

2.2 Displaying Results

In this section, we outline how to use the form data to construct the calls to the RESTful web services of the Tiingo APIs, News API and display the results in the web page. There are basically four components to display in a tabs from on the web page: Company Outlook, Stock Summary, Charts and Latest News.

A sample result is shown in Figure 3.

Figure 3: A Sample Search Result for Apple Inc (AAPL)

2.2.1 Displaying Company Outlook Tab

We use the Tiingo Meta Endpoint API. Please refer to https://api.tiingo.com/documentation/end-of-day for more details on the API.

API Sample: https://api.tiingo.com/tiingo/daily/keyword?token=API_KEY

The process to create your API key is explained in section 3. When constructing the python request required obtaining the Company Details, you need to provide two values in the URL:

1. The first value, the keyword, is the stock ticker the user entered in the form.
2. The second value, the token is your API_KEY that you create as described in section 3. For example, to get information about AAPL, the stock ticker or Apple, you would call the API as: https://api.tiingo.com/tiingo/daily/aapl?token=0108ecd7f84bf97f8f11998a4e4561bx51da92c8

The response of this Python request is a JSON-formatted object. Figure 4 shows a sample response from the request. You need to parse this JSON object and extract some fields as required. The mapping of the JSON response to the tabular data to be displayed on the screen is shown in Table 1.

 

Figure 4: Sample JSON response from Tiingo APIs metadata endpoint

 

Tabular Data	Data from Tiingo Metadata API response JSON
Company Name	The value of key name
Stock Ticker Symbol	The value of key ticker
Stock Exchange Code	The value of key exchangeCode
Company Start Date	The value of key startDate
Description	The value of key description. The description needs to be truncated with an ellipsis (triple dots) so as to only span the first FIVE lines.

 

Table 1: Mapping JSON data and company outlook display

 

After extracting the data, the data should be displayed in a tabular format. A sample output is shown in Figure 5.

 

Figure 5: Example of Company Outlook Tab

 

2.2.2 Displaying Stock Summary Tab

We use the Tiingo Current Top-of-Book & Last Price API to fetch the stock information for the ticker. Please refer to https://api.tiingo.com/documentation/iex for more details on the API.

API Sample: https://api.tiingo.com/iex/keyword?token=API_KEY

When constructing the python request required obtaining the Company Details, you need to provide two values in the URL:

1. The first value, the keyword, is the stock ticker the user entered in the form.
2. The second value, the token is your API_KEY that you create as described in section 3.

For example, to get information about AAPL, the stock ticker or Apple, you would call the API as:

https://api.tiingo.com/iex/aapl?token=0108ecd7f84bf97f8f11998a4e4545ea51da92c8

The response of this Python request is a JSON-formatted object. Figure 6 shows a sample response from the request. You need to parse this JSON object and extract some fields as required. The mapping of the JSON response to the tabular data to be displayed on the screen is shown in Table 2.

Figure 6: Sample JSON response from Tiingo APIs Current Top-of-Book & Last Price endpoint

Tabular Data	Data from Tiingo Current Top-of-Book & Last Price API response JSON
Stock Ticker Symbol	The value of key ticker
Trading Day	The value of the key timestamp. Truncate the value to just display the date.
Previous Closing Price	The value of key prevClose
Opening Price	The value of key open
High Price	The value of key high
Low Price	The value of key low
Last Price	The value of key last
Change	Difference between the value of key last and prevClose. Also display the upward arrow or downward arrow depending on whether the difference value is positive or negative. Reference to the image links in Section 3. Change = (last prevClose)
Change Percent	The Change Percent between the value of key last and prevClose.Also display the upward arrow or downward arrow depending on whether the difference value is positive or negative. Reference to the image in Section 3. Change Percent = (Change / prevClose) * 100 Please refer to below link for formula:https://www.skillsyouneed.com/num/percent-change.html
Number of Shares Traded	The value of key volume

Table 2: Mapping JSON data and stock summary display

 

After extracting the data, the data should be displayed in a tabular format. A sample output is shown in Figure 7.

Figure 7: Example of Stock Summary Information Tab

 

2.2.3 Displaying Stock Price/Volume Chart Tab

You should extract the content of Time Series Data from the returned JSON object to construct a chart which is responsible for displaying (close) price/volume. The chart is provided by HighCharts. Find more information about HighCharts at: https://www.highcharts.com/demo

The historical data for the ticker can be obtained from Tiingos Historical Intraday Prices Endpoint API.

Please refer to https://api.tiingo.com/documentation/iex for more details on the API.

API Sample:

https://api.tiingo.com/iex/keyword/prices?startDate=prior_date&resampleFreq=12hour&columns=o pen,high,low,close,volume&token=API_KEY

When constructing the python request required obtaining the Company Details, you need to provide 5 values:

1. The first value, the keyword, is the stock ticker the user entered in the form.
2. The second value, the token is your API_KEY that you create as described in section 3.
3. The third value is the startDate, a prior date which should be 6 months prior to the current date. You can use Python DateUtilss relativedelta to calculate the date. Please refer to https://dateutil.readthedocs.io/en/stable/relativedelta.html for more details.
4. The fourth value, columns, indicates what you want returned: open, high, low and close prices and volume.
5. The fifth value, resampleFreq, indicates in hours the resampling frequency. See the Tiingo documentation for allowed values.

For example, to get information about AAPL Historical Intraday Prices, you would call the API as: https://api.tiingo.com/iex/aapl/prices?startDate=2020-03-

04&resampleFreq=12hour&columns=open,high,low,close,volume&token=0108ecd7f84bf97f8f1199

8a4e4561ea51da92c8

The response of this Python request is a JSON-formatted object. Figure 8 shows a sample response from the request. You need to parse this JSON object and extract some fields as required.

Figure 8: Sample JSON response from Tiingo APIs Historical Intraday Prices Endpoint

 

 

The data obtained from the API can then be mapped to the HighCharts dataset using the mapping below.

Chart Data	Data from Tiingo Historical Intraday Prices API response JSON
Date	The value of key name
Stock Price	The value of key close
Volume	The value of key volume

 

Table 3: Mapping JSON data and HighCharts data

 

For mapping the Stock price data to data for HighCharts, create an array of data points (x1, y1) where x1 will be the date(in UTC format) and y1 will be the corresponding close stock price for that day. This array will then act as an input dataset for your HighCharts. Please refer to links in Section 3 for more details.

Similarly create another array of points (x2, y2) where x2 will be the date (also in UTC format) and y1 will be the volume for that day. This array will be the second input for your HighCharts. Since you will be plotting Stock Price vs Date and Volume vs Date you will have two different datasets and two yaxis and a single x-axis.

 

Initially, the chart shows the historical stock price (in blue line with filling the area below, two digits after decimal) and volume (in grey bar) for the past six months by an interval of one day. Figure 9 shows an example of the Stock Price/Volume chart.

Figure 9: An Example of Chart showing Stock Price/Volume for 6 months

The title of the chart for showing price/volume is Stock Price <Ticker> (YYYY-MM-DD), where YYYY-MM-DD is todays date. The subtitle of the chart should be Source: Tiingo and should hyperlink to the Tiingo website: https://api.tiingo.com/. The title of the Y-axis is Stock Price when showing the stock price and the other Y-axis is Volume.

The X-axis changes on the basis of the zoom level 6 months, 3 months, 1 month, 15 days, and 7 days. Please refer to Section 3 for references on how to change the chart data on the basis of the zoom level. Figure 10 shows an example of the Stock Price/Volume chart for 15 days zoom level.

Figure 10: An Example of Chart showing Stock Price/Volume for 15 days

 

2.2.4 Displaying Stock News Tab

To display news for the company, we should use the stock ticker symbol from the form data to construct a Python request to the News API and display the results in your page. Please refer to https://newsapi.org/docs for more details.

API Sample: https://newsapi.org/v2/everything?apiKey=API_KEY&q=keyword

When constructing the python request required obtaining the News articles, you need to provide two values:

1. The first value is your apiKey, which you created as described in section 3.
2. The second value, q, is the stock ticker the user entered in the form.

The response of this Python request is a JSON-formatted object. Figure 11 shows a sample response from the request. You need to parse this JSON object and extract some fields as required. Only the articles that have title, url, urlToImage and publishedAt keys present with a non-empty value should be displayed. If any attribute is missing, empty or blank, do not display it in the results. Display the first five articles from the response which have all the keys and values present as mentioned.

Figure 11: Sample JSON response from News APIs Endpoint

 

The mapping between the information populated in the results and the JSON object is shown in Table 4.

Card Data	Data from News API JSON response
Image	The value of key urlToImage
Title	The value of key title
Date	The value of key publishedAt
Link to Original Post	The value of key url

Table 4: Mapping between result card and JSON object

After extracting the data, it should be displayed as shown in Figure 12. The See Original Post hyperlink opens the original post in a new tab in the browser.

 

Figure 12: Example of Latest News

2.2.5 Displaying Error Message

If the Tiingo API service returns an Invalid Call error message, the page should display Error : No record has been found, please enter a valid symbol. Figure 13 shows an example when searching for non-existent stock symbol xyz.

Figure 13: Search Result when there is no matching result

2.3 Saving Previous Inputs

In addition to displaying the results, the page should maintain the entered value while displaying the current result. For example, if a user searches for AAPL, the user should see what was provided in the search form when displaying the results. Specifically, when clicking on the Search button, the page should display the result retrieved from the Tiingo API service and keep the value provided in the search form.

 

 

 

3. Hints

3.1 Tiingo API Documentation

To apply for an API key on Tiingo and read the API documentation, please go to: https://api.tiingo.com/.

Click on the Sign-up button. Enter your Username, your @usc.edu E-mail address, select and confirm a Password, and click on Sign-up. Verify your account and go to https://api.tiingo.com/account/api/token to get your API_KEY.

3.2 Image References

Regarding the marker icon rendered beside the values of Change, Change Percent, a red-down arrow icon is displayed if the value is negative while a green-up arrow icon is displayed if the value is positive. The icons can be found at:

https://csci571.com/hw/hw6/images/GreenArrowUp.jpg https://csci571.com/hw/hw6/images/RedArrowDown.jpg

3.3 Deploy Python file to the cloud (GAE/AWS/Azure)

You should use the domain name of the GAE/AWS/Azure service you created in HW #5 to make the request. For example, if your GAE/AWS/Azure server domain is called example.appspot.com or example.elasticbeanstalk.com or example.azurewebsites.net, the following links will be generated:

GAE http://example.appspot.com/index.html

AWS http://example.elasticbeanstalk.com/index.html

Azure http://example.azurewebsites.net/index.html

The example subdomain in the above URLs will be replaced by your choice of subdomain from the cloud service. You may also use a different page than index.html.

3.4 How to get News API Key

To apply for an API Key for newsapi.org, go to https://newsapi.org/docs.

Click on the Get API key button. You should fill out the form as shown in Figure 14. Click Submit.

The API Key (News API key) will be displayed. Copy this key as you will use it in all the News API calls.

Figure 14: Google News Register for API Key Form

3.5 References for HighCharts

 

1. https://api.highcharts.com/highstock/
2. https://www.highcharts.com/demo/stock/intraday-area
3. https://www.highcharts.com/demo/stock/compare