[SOLVED] python c/c++ Java Microsoft Word streaming1.docx

Microsoft Word streaming1.docx

ProgrammingAssignment#1

COEN317DistributedSystem
DepartmentofComputerEngineering

SantaClaraUniversity

Dr.Ming-HwaWangSpringQuarter2017
Phone:(408)526-4844 Emailaddress:[email protected]
Coursewebsite: http://www.cse.scu.edu/~mwang2/distributed/
OfficeHours:Monday9:30-10:00pm

Duedate:midnightFebruary5,2016

SimplifiedDistributedStreamingPlatform,PartI(200points)
PleaseimplementasimplifieddistributedstreamingplatformusingPython,C/C++or
Javausingclient-servernetworkprogramming.Inadistributedstreamingplatform,
aproducercanpublishastreamofrecordstooneormoretopics,aconsumer(with
a unique name) can subscribe one ormore topics, the platform stores streamof
recordsinareliablefault-tolerantway,andprocess(i.e.,transformtheinputstreams
tooutputstreams)streamofrecordsastheyoccurinaqueueforeachpartitionof
eachtopic.Eachrecordconsistsofakey,avalueandatimestamp.Tosimplifyyour
work,thekeyisastringandthevalueisaninteger,butthereisnoneedtosupport
timestamp.Eachconsumerownsapartitionforoneormoretopicsitsubscribed,and
theplatformwillkeeptheoffsetforeachconsumerandanowoffsetforeachtopic.
If thenumberofconsumers is less thanorequal to thenumberofpartitions, the
consumersshareallpartitions.Ifthenumberofconsumersisequaltothenumberof
partitions, then any new consumer needs towait until other completed.We use
replicationfactoroftwo.PartIofthisassignmentimplementthebasicfunctionsof
the platform with assumption that no server will be down, and part II supports
redundancyandfault-tolerant.

Tosimplifyyourwork,bothserverandclientrunontheLinuxmachines.Youshould
firstrunaserverprogramoneverymachinewhichmayjointheplatform,andthe
serverwill findanavailableportautomaticallyanddisplaythecorrespondingport
number. Then runP1with auniquenameand IP addressof the server,anduse
subcommand(add/deletehosts,publish,subscribe).E.g.,
OnmachinewithIPaddress23.253.20.67,do:
$ server itu_server1
itu_server1 at IP address 23.253.20.67 and port number: 9998

OnmachinewithIPaddress104.130.67.11,do:
$ server itu_server2
itu_server2 at IP address 104.130.67.11 and port number: 3571
Onanymachinedo:
$ client client_1
client_1> add (name=itu_server_1 ip=23.253.20.67 port=9998)
(name=itu_server_2 ip=104.130.67.11 port=3571)
client_1> create (topic=topic_1 partitions=2) (topic=t2)
client_1> subscribe (topic=topic_1)
client_1 subscribed topic_1 and can get partition 0 and 1
client_1>
Onanothermachine,do:
$ client client_2
client_2> subscribe (topic=topic_1) (topic=t2)
client_2 subscribed topic_1 and can get partition 1
client_2 subscribed t2 and can get partition 0
Atthistime,theclient_1shouldgetthenotificationasbelow:
client_1 subscribed topic_1 and can get partition 0
Continueonclient_2,do:
client_2> publish (topic=topic_1 partition=2 key=def value=1)
put (def, 1) to topic_1 on partition 1 on itu_server2
client_2> publish (topic=topic_1 key=abc value=3)
put (abc, 3) to topic_1 on partition 0 on itu_server1 and
partition 1 on itu_server2
client_2> get (topic=topic_1 partition=1)
get (def, 1) from topic1 and partition 1 on itu_server2
client_2> get (topic=t2 partition=0)
Error: topic t2 has no data left
client_2>
Onclient_1,do:
client_1> get (topic=topic_1 partition=0)
get (abc, 3) from topic1 and partition 0 on itu_server1
client_1>

The add hosts will run server program on the hosts and report available port
numbers.Allthoseinformationshouldbesavedonthehosts/tmp/ /stream/
/info and all record should store in /tmp/ /stream/ /data.
Make the /tmp/ , /tmp/ /stream and /tmp/ /stream/
mode 777, and make info and data files mode 666. Which host to store the
topic/partition is decided by hashing, e.g., md5sum <<< , or echo
|md5sum.The isthetopicwithoptionallyconcatenatedwiththe
partitionnumber.

Thecommandareserverandclient,andthesubcommandsare:
1) {( , , )}
2) {(topic= [partition= ])}#default =1
3) {(topic= )}
4) publish(topic= [partition= ])
5) get(topic= partition= )
Where [] means optional, {} means can be one or more times. And we use
= forparameterwithoutworrytheordering.Thedelete,remove,and
unsubscribeisforpartII.

StudentName:

ID:

Score:

Correctnessandboundarycondition(60%):

ErrorHandling(5%):

Automatic available port finding and support both host name and IP address
(5%):

Displayoutputonbothserverandclientwindowswheneverthereisanevent
happens(10%):

Modulardesign,file/directoryorganizing,showinginput,documentation,coding
standards,sympathy/typingpointwithREADME(20%):

Subtotal:

Latepenalty(20%perday):

Specialservicepenalty(5%):

Totalscore: