GRPC and Distributed Systems
Forward
In this project, you will design and implement a simple distributed file system DFS.First, you will develop several file transfer protocols using gRPC and Protocol Buffers. Next, you will incorporate a weakly consistent synchronization system to manage cache consistency between multiple clients and a single server. The system should be able to handle both binary and textbased files.
Your source code will use a combination of C14, gRPC, and Protocol Buffers to complete the implementation.
Setup
You can clone the code in the Project 4 repository with the command:
git clone https:github.gatech.edugiosfall19pr4.git
Submission Instructions
Submit all code via the submit.py script given at the top level of the repository. For instructions on how to submit individual components of the assignment, see the instructions within Part 1docspart1.md and Part 2docspart2.md. Instructions for submitting the readme are in the next section.
For this assignment, you may submit your code up to 10 times in 24 hours. After the deadline, we download your last submission before the deadline, review your submission, and assign a grade. There is no limit to the number of times you may submit your readmestudent.md file.
After submitting, you may doublecheck the results of your submission by visiting the UdacityGT Bonnie websitehttps:bonnie.udacity.com and going to the student portal.
Readme
Throughout the project, we encourage you to keep notes on what you have done, how you have approached each part of the project, and what resources you used in preparing your work. We have provided you with a prototype file, readmestudent.md that you should use throughout the project.
You may submit your readmestudent.md file with the command:
python submit.py readme
At the prompt, please provide your GT username and password.
If this is your first time submitting, the program will then ask you if you want to save the JWT. If you reply yes, it will save a token on your filesystem so that you dont have to provide your username and password each time that you submit.
The Udacity site will store your readmestudent.md file in a database, where it will be used during grading. The submit script will acknowledge receipt of your README file. For this project, like in Project 3, you only need to submit one README file for both parts of the project.
Note: you may choose to submit a PDF version of this file readmestudent.pdf in place of the markdown version. The submission script will automatically detect and submit this file if it is present in the project root directory. If you submit both files, we will give the PDF version preference.
Directions
Directions for Part 1 can be found in docspart1.mddocspart1.md
Directions for Part 2 can be found in docspart2.mddocspart2.md
Log Utility
Weve provided a simple logging utility, dfslog, in this assignment that can be used within your project files.
There are five log levels LLSYSINFO, LLERROR, LLDEBUG, LLDEBUG2, and LLDEBUG3.
During Bonnie tests, only log levels LLSYSINFO, LLERROR, and LLDEBUG will be output. The others will be ignored. You may use the other log levels for your own debugging and testing.
The log utility uses a simple streaming syntax. To use it, make the function call with the log level desired, then stream your messages to it. For example:
dfslogLLDEBUGType your message here: addavariable, add more info, etc.
References
Relevant lecture material
P4L1 Remote Procedure Callshttps:www.udacity.comcourseviewer!cud923l3450238825
gRPC and Protocol Buffer resources
gRPC C Referencehttps:grpc.github.iogrpccppindex.html
Protocol Buffers 3 Language Guidehttps:developers.google.comprotocolbuffersdocsproto3
gRPC C Exampleshttps:github.comgrpcgrpctreemasterexamplescpp
gRPC C Tutorialhttps:grpc.iodocstutorialsbasiccpp
Protobuffers Scalar typeshttps:developers.google.comprotocolbuffersdocsproto3scalar
gRPC Status Codeshttps:github.comgrpcgrpcblobmasterdocstatuscodes.md
gRPC Deadlinehttps:grpc.ioblogdeadlines
Rubric
Your project will be graded at least on the following items:
Interface specification .proto
Service implementation
gRPC initiation and binding
Proper handling of deadline timeouts
Proper clean up of memory and gRPC resources
Proper communication with the server
Proper request and management of write locks
Proper synchronization of files between multiple clients and a single server
Insightful observations in the Readme file and suggestions for improving the class for future semesters
gRPC Implementation 35 points
Full credit requires: code compiles successfully, does not crash, files fully transmitted, basic safety checks, and proper use of gRPC including the ability to get, store, and list files, along with the ability to recognize a timeout. Note that the automated tests will test some of these automatically, but graders may execute additional tests of these requirements.
DFS Implementation 55 points
Full credit requires: code compiles successfully, does not crash, files fully transmitted, basic safety checks, proper use of gRPC, write locks properly handled, cache properly handled, synchronization of sync and inotify threads properly handled, and synchronization of multiple clients to a single server. Note that the automated tests will test some of these automatically, but graders may execute additional tests of these requirements.
README 10 points5 point extra credit opportunity
Clearly demonstrates your understanding of what you did and whywe want to see your design and your explanation of the choices that you made and why you made those choices. 4 points
A description of the flow of control for your code; we strongly suggest that you use graphics here, but a thorough textual explanation is sufficient. 2 points
A brief explanation of how you implemented and tested your code. 2 points
References any external materials that you consulted during your development process 2 points
Suggestions on how you would improve the documentation, sample code, testing, or other aspects of the project up to 5 points extra credit available for noteworthy suggestions here, e.g., actual descriptions of how you would change things, sample code, code for tests, etc. We do not give extra credit for simply reporting an issuewere looking for actionable suggestions on how to improve things.
Questions
For all questions, please use the class Piazza forum or the class Slack channel so that TAs and other students can assist you.
Reviews
There are no reviews yet.