[SOLVED] 5.Neural-Networks

5.Neural-Networks

L

1HXUDO1HWZRUNV

*KRODPUH]D+DbDUL

LL

1HXUDO1HWZRUNV

*KRODPUH]D+DeDUL

*HQHUDWHGE$OH[DQGULDKWWSVZZZDOH[DQGULDUHSRVLWRURUJRQ0DUFKDWSP$(7

$SSHQGL[+2/LEUDU

LLL

&RQWHQWV
7LWOH L
&RSULJKW LL
)HHG)RUZDUG1HXUDO1HWZRUNV
1HWZRUN7UDLQLQJ
5HJXODULVDWLRQ
$FWLYLW1HXUDO1HWZRUN
8QVXSHUYLVHGDQG6HOI7DXJKW/HDUQLQJ
$FWLYLW$XWRHQFRGHU
237,21$/&RQYROXWLRQDO1HXUDO1HWZRUNV
$SSHQGL[+2/LEUDU

L

)HHG)RUZDUG1HXUDO1HWZRUNV

)HHG)RUZDUG1HXUDO1HWZRUNV

7KH%DVLFV
1HXURQ7RGHVFULEHQHXUDOQHWZRUNVZHZLOOEHJLQEGHVFULELQJWKHVLPSOHVWSRVVLEOHQHXUDOQHWZRUN
RQHZKLFKFRPSULVHVDVLQJOHQHXURQ:HZLOOXVHWKHIROORZLQJGLDJUDPWRGHQRWHDVLQJOHQHXURQ

KWWSVZZZDOH[DQGULDUHSRVLWRURUJZSFRQWHQWXSORDGV
6LQJOH1HXURQSQJ
)LJXUH$VLQJOHQHXURQ

7KLVQHXURQLVDFRPSXWDWLRQDOXQLWWKDWWDNHVDVLQSXW DQGDLQWHUFHSWWHUP

DQGRXWSXWV ZKHUH LVFDOOHGWKHDFWLYDWLRQ
IXQFWLRQ

$FWLYDWLRQ)XQFWLRQ:HXVXDOOFKRRVH WREHWKHVLJPRLGIXQFWLRQ

7KHRXWSXWUDQJHRIWKHVLJPRLGIXQFWLRQLV>@$QRWKHUFRPPRQFKRLFHIRU LVWKHKSHUEROLF
WDQJHQWRUWDQKIXQFWLRQ

7KHRXWSXWUDQJHRIWKHWDQKIXQFWLRQLV>@,QWHUPVRIWKHRXWSXWUDQJHWKHWDQKIXQFWLRQLVD
UHVFDOHGYHUVLRQRIWKHVLJPRLGIXQFWLRQ)LJXUH

)HHG)RUZDUG1HXUDO1HWZRUNV

)LJXUH7KUHHDFWLYDWLRQIXQFWLRQVWDQKVLJPRLGDQGUHFWLcHGOLQHDU

/HW
VKDYHDORRNRQWKHGHULYDWLYHVRIWKHVHDFWLYDWLRQVIXQFWLRQVVLQFHZHZLOOXVHWKHPIRUWKHWUDLQLQJ
RIWKHPRGHOH[SODLQHGLQWKHQH[WFKDSWHU

1HXUDO1HWZRUNV
1HXUDO1HWZRUN0RGHO$QHXUDOQHWZRUNLVSXWWRJHWKHUEKRRNLQJWRJHWKHUPDQRIRXUVLPSOH
QHXURQVVRWKDWWKHRXWSXWRIDQHXURQFDQEHWKHLQSXWRIDQRWKHU)RUH[DPSOHKHUHLVDVPDOOQHXUDO
QHWZRUN

)HHG)RUZDUG1HXUDO1HWZRUNV

KWWSVZZZDOH[DQGULDUHSRVLWRURUJZSFRQWHQWXSORDGV1HWZRUNSQJ
)LJXUH$ODHUQHXUDOQHWZRUN

,QWKLVfJXUHZHKDYHXVHGFLUFOHVWRDOVRGHQRWHWKHLQSXWVWRWKHQHWZRUN7KHFLUFOHVODEHOHGDUH
FDOOHGELDVXQLWVDQGFRUUHVSRQGWRWKHLQWHUFHSWWHUP7KHOHIWPRVWODHURIWKHQHWZRUNLVFDOOHGWKH
LQSXWODHUDQGWKHULJKWPRVWODHUWKHRXWSXWODHUZKLFKLQWKLVH[DPSOHKDVRQORQHQRGH7KH
PLGGOHODHURIQRGHVLVFDOOHGWKHKLGGHQODHUEHFDXVHLWVYDOXHVDUHQRWREVHUYHGLQWKHWUDLQLQJVHW
:HDOVRVDWKDWRXUH[DPSOHQHXUDOQHWZRUNKDVLQSXWXQLWVQRWFRXQWLQJWKHELDVXQLWKLGGHQ
XQLWVDQGRXWSXWXQLW

1HWZRUN3DUDPHWHUV/HW
VGHQRWHWKHQXPEHURIODHUVLQRXUQHWZRUNE WKXV LQRXU

H[DPSOH/HW
VODEHOODHU DV VRODHU LVWKHLQSXWODHUDQGODHU WKHRXWSXWODHU2XU

QHXUDOQHWZRUNKDVSDUDPHWHUV ZKHUHZHZULWH WRGHQRWHWKH
SDUDPHWHURUZHLJKWDVVRFLDWHGZLWKWKHFRQQHFWLRQEHWZHHQXQLW LQODHU DQGXQLW LQODHU

QRWHWKHRUGHURIWKHLQGLFHV$OVR LVWKHELDVDVVRFLDWHGZLWKXQLW LQODHU 7KXVLQRXU

H[DPSOHZHKDYH DQG 1RWHWKDWELDVXQLWVGRQ
WKDYHLQSXWVRU
FRQQHFWLRQVJRLQJLQWRWKHPVLQFHWKHDOZDVRXWSXWWKHYDOXH:HDOVROHW GHQRWHWKHQXPEHURI
QRGHVLQODHU QRWFRXQWLQJWKHELDVXQLW

1HXURQ
V$FWLYDWLRQ/HW GHQRWHWKHWRWDOZHLJKWHGVXPRILQSXWVWRXQLW LQODHU LQFOXGLQJWKH

ELDVWHUPHJ 7KHQWKHRXWSXWRIWKHQHXWURQDNDLWVDFWLYDWLRQLV

)HHG)RUZDUG1HXUDO1HWZRUNV

GHQRWHGE

)RUZDUG3URSDJDWLRQ*LYHQDf[HGVHWWLQJRIWKHSDUDPHWHUV RXUQHXUDOQHWZRUNGHfQHVDIXQFWLRQ

WKDWRXWSXWVDUHDOQXPEHU6SHFLfFDOOWKHFRPSXWDWLRQWKDWRXUH[DPSOHQHXUDOQHWZRUN
UHSUHVHQWVLVJLYHQE

1RWHWKDWWKLVHDVLOOHQGVLWVHOIWRDPRUHFRPSDFWQRWDWLRQ6SHFLfFDOOLIZHH[WHQGWKHDFWLYDWLRQ

IXQFWLRQ WRDSSOWRYHFWRUVLQDQHOHPHQWZLVHIDVKLRQLH

WKHQZHFDQZULWHWKHHTXDWLRQVDERYHPRUHFRPSDFWO
DV

:HFDOOWKLVVWHSIRUZDUGSURSDJDWLRQ0RUHJHQHUDOOUHFDOOLQJWKDWZHXVH WRDOVRGHQRWHWKH

YDOXHVIURPWKHLQSXWODHUWKHQJLYHQODHU
VDFWLYDWLRQV ZHFDQFRPSXWHODHU
V

DFWLYDWLRQV DV

%RUJDQL]LQJRXUSDUDPHWHUVLQPDWULFHVDQGXVLQJPDWUL[YHFWRURSHUDWLRQVHJSURYLGHGE*38VZH
FDQWDNHDGYDQWDJHRIIDVWOLQHDUDOJHEUDURXWLQHVWRTXLFNOSHUIRUPFDOFXODWLRQVLQRXUQHWZRUN

)HHG)RUZDUG1HXUDO1HWZRUN:HKDYHVRIDUIRFXVHGRQRQHH[DPSOHQHXUDOQHWZRUNEXWRQHFDQ
DOVREXLOGQHXUDOQHWZRUNVZLWKRWKHUDUFKLWHFWXUHVPHDQLQJSDWWHUQVRIFRQQHFWLYLWEHWZHHQQHXURQV
LQFOXGLQJRQHVZLWKPXOWLSOHKLGGHQODHUV7KHPRVWFRPPRQFKRLFHLVD ODHUHGQHWZRUNZKHUHODHU
LVWKHLQSXWODHUODHU LVWKHRXWSXWODHUDQGHDFKODHU LVGHQVHOFRQQHFWHGWRODHU ,Q

WKLVVHWWLQJWRFRPSXWHWKHRXWSXWRIWKHQHWZRUNZHFDQVXFFHVVLYHOFRPSXWHDOOWKHDFWLYDWLRQVLQ

ODHU WKHQODHU DQGVRRQXSWRODHU XVLQJWKHHTXDWLRQVDERYHWKDWGHVFULEHWKHIRUZDUG
SURSDJDWLRQVWHS7KLVLVRQHH[DPSOHRIDIHHGIRUZDUGQHXUDOQHWZRUNVLQFHWKHFRQQHFWLYLWJUDSKGRHV
QRWKDYHDQGLUHFWHGORRSVRUFFOHV

)HHG)RUZDUG1HXUDO1HWZRUNV

1HXUDOQHWZRUNVFDQDOVRKDYHPXOWLSOHRXWSXWXQLWV)RUH[DPSOHKHUHLVDQHWZRUNZLWKWZRKLGGHQ
ODHUVODHUV DQG DQGWZRRXWSXWXQLWVLQODHU

KWWSVZZZDOH[DQGULDUHSRVLWRURUJZSFRQWHQWXSORDGV1HWZRUNSQJ
)LJXUH$ODHUQHXUDOQHWZRUNZLWKWZRXQLWVDWWKHRXWSXWODHU

7RWUDLQWKLVQHWZRUNZHZRXOGQHHGWUDLQLQJH[DPSOHV ZKHUH 7KLVVRUWRIQHWZRUN
LVXVHIXOLIWKHUHDUHPXOWLSOHRXWSXWVWKDWRXDUHLQWHUHVWHGLQSUHGLFWLQJ)RUH[DPSOHLQDPHGLFDO
GLDJQRVLVDSSOLFDWLRQWKHYHFWRU PLJKWJLYHWKHLQSXWIHDWXUHVRIDSDWLHQWDQGWKHGLeHUHQWRXWSXWV

VPLJKWLQGLFDWHSUHVHQFHRUDEVHQFHRIGLeHUHQWGLVHDVHV

11VYV3HUFHSWURQ$VFDQEHVHHQIURPWKHDERYHDUFKLWHFWXUHVWKHQHXUDOQHWZRUNPRGHOFRPSULVHV
PXOWLSOHVWDJHVRISURFHVVLQJHDFKRIZKLFKUHVHPEOHVWKHSHUFHSWURQPRGHORI0RGXOHDQGIRUWKLV
UHDVRQWKHQHXUDOQHWZRUNLVDOVRNQRZQDVWKHPXOWLODHUSHUFHSWURQRU0/3$NHGLeHUHQFHFRPSDUHG
WRWKHSHUFHSWURQKRZHYHULVWKDWWKHQHXUDOQHWZRUNXVHVFRQWLQXRXVQRQOLQHDULWLHVLQWKHKLGGHQXQLWV

HJ RU ZKHUHDVWKHSHUFHSWURQXVHVVWHSIXQFWLRQQRQOLQHDULWLHV7KLVPHDQVWKDWWKH
QHXUDOQHWZRUNIXQFWLRQLVGLeHUHQWLDEOHZLWKUHVSHFWWRWKHQHWZRUNSDUDPHWHUVDQGWKLVSURSHUWZLOO
SODDFHQWUDOUROHLQQHWZRUNWUDLQLQJZKLFKZHZLOOVHHLQWKHQH[W&KDSWHU

7KH3RZHURI1HXUDO1HWZRUNV
7KHDSSUR[LPDWLRQSURSHUWLHVRIIHHGIRUZDUGQHWZRUNVKDYHEHHQZLGHOVWXGLHGDQGIRXQGWREHYHU
JHQHUDOWKHPRGHOFODVVFRUUHVSRQGLQJWRQHXUDOQHWZRUNVFDQUHSUHVHQWDOPRVWDQIXQFWLRQJLYHQ
VRPHPLQRUFRQGLWLRQV1HXUDOQHWZRUNVDUHWKHUHIRUHVDLGWREHXQLYHUVDODSSUR[LPDWRUV)RUH[DPSOHD
WZRODHUQHWZRUNZLWKOLQHDURXWSXWVFDQXQLIRUPODSSUR[LPDWHDQFRQWLQXRXVIXQFWLRQRQDQLQSXW
GRPDLQWRDUELWUDUDFFXUDFSURYLGHGWKHQHWZRUNKDVDVXhFLHQWOODUJHQXPEHURIKLGGHQXQLWV7KLV
UHVXOWKROGVIRUDZLGHUDQJHRIKLGGHQXQLWDFWLYDWLRQIXQFWLRQVEXWH[FOXGLQJSROQRPLDOV

,QWKHFDVHRIFODVVLfFDWLRQSUREOHPVWKLVUHVXOWHQVXUHVWKDWQHXUDOQHWZRUNVFDQDSSUR[LPDWHWKHWDUJHW
GHFLVLRQERXQGDUWRDQUHTXLUHGSUHFLVLRQ6LPLODUOLQWKHFDVHRIUHJUHVVLRQSUREOHPVWKLVUHVXOW
HQVXUHVWKDWWKHQHXUDOQHWZRUNPRGHOFDQDSSUR[LPDWHWKHWDUJHWIXQFWLRQWRDQSUHFLVLRQ+RZHYHU

)HHG)RUZDUG1HXUDO1HWZRUNV

WKHSULFHWRSDLQWKHVHFDVHVLVWKHQXPEHURIQHXWURQVLQWKHKLGGHQODHUDVWKHQXPEHURIQHXURQV
JURZVWKHQXPEHURISDUDPHWHUVJURZVDQGWKHQHWZRUNKDVPRUHWHQGHQFWRRYHUfWWKHWUDLQLQJGDWD
7KHRYHUfWWLQJFDQEHSUHYHQWHGEXVLQJDODUJHWUDLQLQJGDWDXVLQJSURSHUUHJXODULVDWLRQ
PHWKRGVDQGXVLQJGHHSDUFKLWHFWXUHZKHUHWKHQHWZRUNFDQUHSUHVHQWWKHVDPHPRGHOFODVVDVD
VKDOORZDUFKLWHFWXUHEXWZLWKDPXFKVPDOOHUQXPEHURIQHXURQV

1HWZRUN7UDLQLQJ

1HWZRUN7UDLQLQJ

7KH7UDLQLQJ2EMHFWLYH
5HJUHVVLRQ6RIDUZHKDYHYLHZHGQHXUDOQHWZRUNVDVDJHQHUDOFODVVRISDUDPHWULFQRQOLQHDUIXQFWLRQV
IURPDYHFWRU RILQSXWYDULDEOHVWRDYHFWRU RIRXWSXWYDULDEOHV$VLPSOHDSSURDFKWRWKHSUREOHPRI
GHWHUPLQLQJWKHQHWZRUNSDUDPHWHUVLVWRPDNHDQDQDORJZLWKWKHGLVFXVVLRQRISROQRPLDOFXUYHfWWLQJ
LQ0RGXOHDQGWKHUHIRUHWRPLQLPL]HDVXPRIVTXDUHVHUURUIXQFWLRQ*LYHQDWUDLQLQJVHWFRPSULVLQJD

VHWRILQSXWYHFWRUV ZKHUH WRJHWKHUZLWKDFRUUHVSRQGLQJVHWRIWDUJHWYHFWRUV

ZHPLQLPL]HWKHHUURUIXQFWLRQ

:HFDQPRWLYDWHWKLVHUURUIXQFWLRQXVLQJDSUREDELOLVWLFLQWHUSUHWDWLRQRIWKHQHWZRUNRXWSXWV
3DUWLFXODUOOHW
VDVVXPHWKDWWKHWDUJHWLVGLVWULEXWHGDFFRUGLQJWRWKHIROORZLQJ*DXVVLDQGLVWULEXWLRQ

ZKHUH LVWKHHHWKHLGHQWLWFRYDULDQFHPDWUL[7KHORJOLNHOLKRRGRIWKHGDWDWKHQLV

6RPD[LPLVLQJWKHORJOLNHOLKRRGIXQFWLRQ LVHTXLYDOHQWWRPLQLQJWKHHUURUIXQFWLRQ 7KH
SUREDELOLVWLFLQWHUSUHWDWLRQSURYLGHVXVZLWKDFOHDUHUPRWLYDWLRQERWKIRUWKHFKRLFHRIRXWSXWXQLW
QRQOLQHDULWDQGWKHFKRLFHRIHUURUIXQFWLRQ

%LQDU&ODVVLcFDWLRQ1RZFRQVLGHUWKHFDVHRIELQDUFODVVLfFDWLRQLQZKLFKZHKDYHDVLQJOHWDUJHW
YDULDEOH VXFKWKDW GHQRWHVFODVV DQG GHQRWHVFODVV :HFRQVLGHUDQHWZRUN

KDYLQJDVLQJOHRXWSXWZKRVHDFWLYDWLRQIXQFWLRQLVDORJLVWLFVLJPRLG :HFDQWKHQLQWHUSUHW

WKHQHWZRUNRXWSXW DVWKHFRQGLWLRQDOSUREDELOLW ZLWK 7KH

FRQGLWLRQDOSUREDELOLWRIDWUDLQLQJLQSXWWDUJHWSDLU LVWKHQ

1HWZRUN7UDLQLQJ

,IZHFRQVLGHUDWUDLQLQJVHWRILQGHSHQGHQWREVHUYDWLRQVWKHQWKHHUURUIXQFWLRQZKLFKLVJLYHQEWKH
QHJDWLYHORJOLNHOLKRRGLVWKHQDFURVVHQWURSHUURUIXQFWLRQRIWKHIRUP

0XOWLFODVV&ODVVLcFDWLRQ)LQDOOZHFRQVLGHUWKHVWDQGDUGPXOWLFODVVFODVVLfFDWLRQSUREOHPLQZKLFK

HDFKLQSXWLVDVVLJQHGWRRQHRI PXWXDOOH[FOXVLYHFODVVHV7KHELQDUWDUJHWYDULDEOHV
KDYHDRI FRGLQJVFKHPHLQGLFDWLQJWKHFODVVDQGWKHQHWZRUNRXWSXWVDUHLQWHUSUHWHGDV

OHDGLQJWRWKHIROORZLQJHUURUIXQFWLRQ

,QWKLVFDVHWKHRXWSXWXQLWDFWLYDWLRQIXQFWLRQLVJLYHQEWKHVRIWPD[IXQFWLRQ

ZKLFKVDWLVfHV DQG 5HFDOOWKDW LVWKHWRWDOZHLJKWHGVXPRILQSXWVWR
WKH WKQHXURQHRIWKHODVWODHU

6XPPDU7KHUHLVDQDWXUDOFKRLFHRIERWKRXWSXWXQLWDFWLYDWLRQIXQFWLRQDQGPDWFKLQJHUURUIXQFWLRQ
DFFRUGLQJWRWKHWSHRISUREOHPEHLQJVROYHG)RUUHJUHVVLRQZHXVHOLQHDURXWSXWVDQGDVXPRIVTXDUHV
HUURUIRUELQDUFODVVLfFDWLRQVZHXVHWKHVLJPRLGRXWSXWVDQGDFURVVHQWURSHUURUIXQFWLRQDQGIRU
PXOWLFODVVFODVVLfFDWLRQZHXVHVRIWPD[RXWSXWVZLWKWKHFRUUHVSRQGLQJPXOWLFODVVFURVVHQWURSHUURU
IXQFWLRQ)RUFODVVLfFDWLRQSUREOHPVLQYROYLQJWZRFODVVHVZHFDQXVHDVLQJOHORJLVWLFVLJPRLGRXWSXWRU
DOWHUQDWLYHOZHFDQXVHDQHWZRUNZLWKWZRRXWSXWVKDYLQJDVRIWPD[RXWSXWDFWLYDWLRQIXQFWLRQ

3DUDPHWHU2SWLPLVDWLRQ
7KH(UURU6XUIDFH:HWXUQQH[WWRWKHWDVNRIfQGLQJDZHLJKWYHFWRU ZKLFKPLQLPL]HVWKHFKRVHQ

HUURUIXQFWLRQ $WWKLVSRLQWLWLVXVHIXOWRKDYHDJHRPHWULFDOSLFWXUHRIWKHHUURUIXQFWLRQZKLFK
ZHFDQYLHZDVDVXUIDFHVLWWLQJRYHUZHLJKWVSDFHDVVKRZQLQ)LJXUH

1HWZRUN7UDLQLQJ

)LJXUH*HRPHWULFDOYLHZRIWKHHUURUIXQFWLRQ$ORFDODQGWKHJOREDOPLQLPXPVDUH
PDUNHG

1RWHLQ)LJXUHWKDWWKHHUURUIXQFWLRQLVDQRQOLQHDUDQGQRQFRQYH[IXQFWLRQRIWKHZHLJKWVZLWK
ORWVRIORFDORSWLPD)XUWKHUPRUHDVSUHYLRXVPRGXOHVZHUHVRUWWRWKHVWRFKDVWLFJUDGLHQWGHVFHQW
DOJRULWKPDVWKHUHLVQRKRSHWRfQGDQDQDOWLFDOVROXWLRQWRWKHRSWLPLVDWLRQSUREOHP

7KH*UDGLHQWHVFHQW$OJRULWKP,QWKHUHVWRIWKLVFKDSWHUZHIRFXVRQPLQLPLVLQJWKHHUURUIXQFWLRQ
IRUWKHUHJUHVVLRQSUREOHPEXWWKHGLVFXVVLRQFDQEHHDVLODSSOLHGWRWKHFODVVLfFDWLRQSUREOHPV/HW
V
GHQRWHWKHFRVWIXQFWLRQWRPLQLPLVHIRUWKHUHJUHVVLRQSUREOHPE

7KHfUVWWHUPLQWKHWKHDYHUDJHVXPRIVTXDUHVHUURUWHUP7KHVHFRQGWHUPLVWKH UHJXODULVDWLRQ
WHUPDOVRFDOOHGDZHLJKWGHFDWHUPWKDWWHQGVWRGHFUHDVHWKHPDJQLWXGHRIWKHZHLJKWVDQGKHOSV

SUHYHQWRYHUfWWLQJ8VXDOOZHLJKWGHFDLVQRWDSSOLHGWRWKHELDVWHUPV DVUHgHFWHGLQRXU

GHfQLWLRQIRU 7KHZHLJKWGHFDSDUDPHWHU FRQWUROVWKHUHODWLYHLPSRUWDQFHRIWKHWZRWHUPV

1HWZRUN7UDLQLQJ

7RXVHJUDGLHQWGHVFHQWDOJRULWKPWRWUDLQRXUQHXUDOQHWZRUNZHQHHGWRLQLWLDOLVHHDFKSDUDPHWHU

DQGHDFK WRDVPDOOUDQGRPYDOXHQHDU]HURVDDFFRUGLQJWRD GLVWULEXWLRQIRUVRPH
VPDOO VDDQGWKHQDSSODQRSWLPL]DWLRQDOJRULWKPVXFKDVVWRFKDVWLFJUDGLHQWGHVFHQW6LQFH

LVDQRQFRQYH[IXQFWLRQJUDGLHQWGHVFHQWLVVXVFHSWLEOHWRORFDORSWLPDKRZHYHULQSUDFWLFH
JUDGLHQWGHVFHQWXVXDOOZRUNVIDLUOZHOO)LQDOOQRWHWKDWLWLVLPSRUWDQWWRLQLWLDOL]HWKHSDUDPHWHUV
UDQGRPOUDWKHUWKDQWRDOO
V,IDOOWKHSDUDPHWHUVVWDUWReDWLGHQWLFDOYDOXHVWKHQDOOWKHKLGGHQODHU

XQLWVZLOOHQGXSOHDUQLQJWKHVDPHIXQFWLRQRIWKHLQSXWPRUHIRUPDOO ZLOOEHWKHVDPHIRUDOO

YDOXHVRI VRWKDW IRUDQLQSXW 7KHUDQGRPLQLWLDOL]DWLRQVHUYHVWKH
SXUSRVHRIVPPHWUEUHDNLQJ

2QHLWHUDWLRQRIJUDGLHQWGHVFHQWXSGDWHVWKHSDUDPHWHUVDVIROORZV

ZKHUH LVWKHOHDUQLQJUDWH7KHNHVWHSLVFRPSXWLQJWKHSDUWLDOGHULYDWLYHVDERYH:HZLOOQRZ
GHVFULEHWKHEDFNSURSDJDWLRQDOJRULWKPZKLFKJLYHVDQHhFLHQWZDWRFRPSXWHWKHVHSDUWLDO
GHULYDWLYHV

7KH%DFNSURSDJDWLRQ$OJRULWKP

*UDGLHQWRIWKH7UDLQLQJ2EMHFWLYH/HW
V :HfUVWGHVFULEHKRZ

EDFNSURSDJDWLRQFDQEHXVHGWRFRPSXWH DQG WKDWLVWKH
SDUWLDOGHULYDWLYHVIRUWKHFRQWULEXWLRQRIRQHWUDLQLQJGDWDSRLQWWRWKHWUDLQLQJREMHFWLYH2QFHZHFDQ

FRPSXWHWKHVHZHVHHWKDWWKHGHULYDWLYHRIWKHRYHUDOOFRVWIXQFWLRQ FDQEHFRPSXWHGDV

*UDGLHQWRID7UDLQLQJDWXP7KHLQWXLWLRQRIWKHEDFNSURSDJDWLRQDOJRULWKPWRFRPSXWHWKHJUDGLHQW

LVDVIROORZV*LYHQDWUDLQLQJGDWXP ZHZLOOfUVWUXQDIRUZDUGSDVVWRFRPSXWHDOOWKHDFWLYDWLRQV

1HWZRUN7UDLQLQJ

WKURXJKRXWWKHQHWZRUNLQFOXGLQJWKHRXWSXWYDOXHRIWKHQHWZRUN 7KHQIRUHDFKQRGH LQODHU

ZHZRXOGOLNHWRFRPSXWHDQHUURUWHUP WKDWPHDVXUHVKRZPXFKWKDWQRGHZDVUHVSRQVLEOHIRU
DQHUURUVLQWKHRXWSXW)RUDQRXWSXWQRGHZHFDQGLUHFWOPHDVXUHWKHGLeHUHQFHEHWZHHQWKH

QHWZRUN
VDFWLYDWLRQDQGWKHWUXHWDUJHWYDOXHDQGXVHWKDWWRGHfQH ZKHUHODHU LVWKHRXWSXW

ODHU+RZDERXWKLGGHQXQLWV)RUWKRVHZHZLOOFRPSXWH EDVHGRQDZHLJKWHGDYHUDJHRIWKHHUURU

WHUPVRIWKHQRGHVWKDWXVHV DVDQLQSXW

7KH%DFNSURSDJDWLRQ$OJRULWKP0RUHVSHFLfFDOOKHUHLVWKHEDFNSURSDJDWLRQDOJRULWKP

3HUIRUPDIHHGIRUZDUGSDVVFRPSXWLQJWKHDFWLYDWLRQVIRUODHUV
)RUHDFKRXWSXWXQLW LQODHU WKHRXWSXWODHUVHW

)RU
)RUHDFKQRGH LQODHU

6HW
&RPSXWHWKHGHVLUHGSDUWLDOGHULYDWLYHVDVIROORZV

)LQDOOZHFDQDOVRUHZULWHWKHDOJRULWKPXVLQJPDWUL[YHFWRULDOQRWDWLRQ

3HUIRUPDIHHGIRUZDUGSDVVFRPSXWLQJWKHDFWLYDWLRQVIRUODHUV
)RUHDFKRXWSXWXQLW LQODHU WKHRXWSXWODHUVHW

)RU
)RUHDFKQRGH LQODHU

6HW
&RPSXWHWKHGHVLUHGSDUWLDOJUDGLHQWVDVIROORZV

ZKHUH GHQRWHVWKHHOHPHQWZLVHLH+DGDPDUGPXOWLSOLFDWLRQEHWZHHQWZRYHFWRUV LI
DQGRQOLI

1HWZRUN7UDLQLQJ

3XWWLQJLW$OO7RJHWKHU:HDUHQRZUHDGWRGHVFULEHWKHIXOOJUDGLHQWGHVFHQWDOJRULWKP,QWKH

SVHXGRFRGHEHORZ LVDPDWUL[RIWKHVDPHGLPHQVLRQDV DQG LVDYHFWRURI

WKHVDPHGLPHQVLRQDV 1RWHWKDWLQWKLVQRWDWLRQ LVDPDWUL[DQGLQSDUWLFXODULWLVQ
W

WLPHV

,QLWLDOLVHUDQGRPOWKHSDUDPHWHUVRIWKHQHWZRUN DQG IRUDOO
:KLOHDVWRSSLQJFRQGLWLRQLVQRWPHWGR

6HW DQG IRUDOO
)RU GR

8VHEDFNSURSDJDWLRQWRFRPSXWH DQG

6HW

6HW

8SGDWHWKHSDUDPHWHUV DQG

7KHDERYHDOJRULWKPLVWKHEDWFKJUDGLHQWGHVFHQWDOJRULWKPWKHSDUDPHWHUVDUHXSGDWHGRQFHDIWHUWKH
JUDGLHQWRIWKHREMHFWLYHIXQFWLRQLVFRPSXWHGRYHUWKHZKROHWUDLQLQJGDWDVHW:HFDQPRGLIWKHLWWR
WKHVWRFKDVWLFJUDGLHQWGHVFHQWDOJRULWKPZKHUHWKHSDUDPHWHUVDUHXSGDWHGLPPHGLDWHODIWHUWKH
JUDGLHQWRIWKHREMHFWLYHIXQFWLRQLVFRPSXWHGIRUHDFKLQGLYLGXDOWUDLQLQJGDWXPLQWKHGDWDVHW7KHUHDUH
DORWRIVWXGLHVLQYHVWLJDWLQJWKHFRQYHUJHQFHEHKDYLRXURIWKHVHWZRgDYRXUVRIWKHJUDGLHQWGHVFHQW
DOJRULWKPOHDQLQJWRZDUGVWKHVXSHULRULWRIWKHVWRFKDVWLFJUDGLHQWGHVFHQWSDUWLFXODUOIRUELJGDWD

5HJXODULVDWLRQ

5HJXODULVDWLRQ

5HJXODULVDWLRQ
7KHQXPEHURILQSXWDQGRXWSXWVXQLWVLQDQHXUDOQHWZRUNLVJHQHUDOOGHWHUPLQHGEWKHGLPHQVLRQDOLW
RIWKHGDWDVHWZKHUHDVWKHQXPEHU RIKLGGHQXQLWVLVDIUHHSDUDPHWHUWKDWFDQEHDGMXVWHGWRJLYH
WKHEHVWSUHGLFWLYHSHUIRUPDQFH1RWHWKDW FRQWUROVWKHQXPEHURISDUDPHWHUVZHLJKWVDQGELDVHVLQ
WKHQHWZRUNDQGVRZHPLJKWH[SHFWWKDWLQDPD[LPXPOLNHOLKRRGVHWWLQJWKHUHZLOOEHDQRSWLPXPYDOXH
RI WKDWJLYHVWKHEHVWJHQHUDOL]DWLRQSHUIRUPDQFHFRUUHVSRQGLQJWRWKHRSWLPXPEDODQFHEHWZHHQ
XQGHUfWWLQJDQGRYHUfWWLQJ)LJXUHVKRZVDQH[DPSOHRIWKHHeHFWRIGLeHUHQWYDOXHVRI IRU
WKHVLQXVRLGDOUHJUHVVLRQSUREOHP

)LJXUH([DPSOHVRIWZRODHUQHWZRUNVWUDLQHGRQGDWDSRLQWVGUDZQIURPWKHVLQXVRLGDOGDWDVHW7KH
JUDSKVVKRZWKHUHVXOWRIcWWLQJQHWZRUNVKDYLQJ0 DQGKLGGHQXQLWVUHVSHFWLYHOEPLQLPL]LQJDVXPRI
VTXDUHVHUURUIXQFWLRQXVLQJDVFDOHGFRQMXJDWHJUDGLHQWDOJRULWKP

7KHJHQHUDOL]DWLRQHUURUKRZHYHULVQRWDVLPSOHIXQFWLRQRI GXHWRWKHSUHVHQFHRIORFDOPLQLPDLQ
WKHHUURUIXQFWLRQDVLOOXVWUDWHGLQ)LJXUH+HUHZHVHHWKHHeHFWRIFKRRVLQJPXOWLSOHUDQGRP
LQLWLDOL]DWLRQVIRUWKHZHLJKWYHFWRUIRUDUDQJHRIYDOXHVRI 7KHRYHUDOOEHVWYDOLGDWLRQVHW
SHUIRUPDQFHLQWKLVFDVHRFFXUUHGIRUDSDUWLFXODUVROXWLRQKDYLQJ ,QSUDFWLFHRQHDSSURDFKWR
FKRRVLQJ LVLQIDFWWRSORWDJUDSKRIWKHNLQGVKRZQLQ)LJXUHDQGWKHQWRFKRRVHWKHVSHFLfF
VROXWLRQKDYLQJWKHVPDOOHVWYDOLGDWLRQVHWHUURU

5HJXODULVDWLRQ

)LJXUH3ORWRIWKHVXPRIVTXDUHVWHVWVHWHUURUIRUWKHSROQRPLDOGDWDVHWYHUVXVWKHQXPEHURIKLGGHQXQLWVLQ
WKHQHWZRUNZLWKUDQGRPVWDUWVIRUHDFKQHWZRUNVL]HVKRZLQJWKHHbHFWRIORFDOPLQLPD)RUHDFKQHZVWDUWWKH
ZHLJKWYHFWRUZDVLQLWLDOL]HGEVDPSOLQJIURPDQLVRWURSLF*DXVVLDQGLVWULEXWLRQKDYLQJDPHDQRI]HURDQGD
YDULDQFHRI

:HLJKWHFD7KHUHDUHKRZHYHURWKHUZDVWRFRQWUROWKHFRPSOH[LWRIDQHXUDOQHWZRUNPRGHOLQ
RUGHUWRDYRLGRYHUfWWLQJ)URPRXUGLVFXVVLRQRISROQRPLDOFXUYHfWWLQJLQ0RGXOHZHVHHWKDWDQ
DOWHUQDWLYHDSSURDFKLVWRFKRRVHDUHODWLYHOODUJHYDOXHIRU DQGWKHQWRFRQWUROFRPSOH[LWEWKH
DGGLWLRQRIDUHJXODUL]DWLRQWHUPWRWKHHUURUIXQFWLRQ7KHVLPSOHVWUHJXODUL]HULVWKHTXDGUDWLFDND
ZHLJKWGHFDJLYLQJDUHJXODULVHGHUURUIXQFWLRQDVZHVDZLQWKHSUHYLRXV&KDSWHU7KHHeHFWLYHPRGHO
FRPSOH[LWLVWKHQGHWHUPLQHGEWKHFKRLFHRIWKHUHJXODUL]DWLRQFRHhFLHQW

(DUO6WRSSLQJ$QDOWHUQDWLYHWRUHJXODULVDWLRQDVDZDRIFRQWUROOLQJWKHHeHFWLYHFRPSOH[LWRID
QHWZRUNLVWKHSURFHGXUHRIHDUOVWRSSLQJ7KHWUDLQLQJRIQRQOLQHDUQHWZRUNPRGHOVFRUUHVSRQGVWRDQ
LWHUDWLYHUHGXFWLRQRIWKHHUURUIXQFWLRQGHfQHGZLWKUHVSHFWWRDVHWRIWUDLQLQJGDWD)RUPDQRIWKH
RSWLPL]DWLRQDOJRULWKPVXVHGIRUQHWZRUNWUDLQLQJVXFKDVEDWFKJUDGLHQWGHVFHQWWKHHUURULVD
QRQLQFUHDVLQJIXQFWLRQRIWKHLWHUDWLRQLQGH[+RZHYHUWKHHUURUPHDVXUHGZLWKUHVSHFWWRLQGHSHQGHQW
GDWDJHQHUDOOFDOOHGDYDOLGDWLRQVHWRIWHQVKRZVDGHFUHDVHDWfUVWIROORZHGEDQLQFUHDVHDVWKH
QHWZRUNVWDUWVWRRYHUfW7UDLQLQJFDQWKHUHIRUHEHVWRSSHGDWWKHSRLQWRIVPDOOHVWHUURUZLWKUHVSHFWWR
WKHYDOLGDWLRQGDWDVHWDVLQGLFDWHGLQ)LJXUHLQRUGHUWRREWDLQDQHWZRUNKDYLQJJRRG
JHQHUDOL]DWLRQSHUIRUPDQFH

5HJXODULVDWLRQ

)LJXUH$QLOOXVWUDWLRQRIWKHEHKDYLRXURIWUDLQLQJVHWHUURUOHIWDQGYDOLGDWLRQVHWHUURUULJKWGXULQJDWSLFDO
WUDLQLQJVHVVLRQDVDIXQFWLRQRIWKHLWHUDWLRQVWHSIRUWKHVLQXVRLGDOGDWDVHW7KHJRDORIDFKLHYLQJWKHEHVW
JHQHUDOL]DWLRQSHUIRUPDQFHVXJJHVWVWKDWWUDLQLQJVKRXOGEHVWRSSHGDWWKHSRLQWVKRZQEWKHYHUWLFDOGDVKHGOLQHV
FRUUHVSRQGLQJWRWKHPLQLPXPRIWKHYDOLGDWLRQVHWHUURU

7KHEHKDYLRXURIWKHQHWZRUNLQWKLVFDVHLVVRPHWLPHVH[SODLQHGTXDOLWDWLYHOLQWHUPVRIWKHHeHFWLYH
QXPEHURIGHJUHHVRIIUHHGRPLQWKHQHWZRUNLQZKLFKWKLVQXPEHUVWDUWVRXWVPDOODQGWKHQWRJURZV
GXULQJWKHWUDLQLQJSURFHVVFRUUHVSRQGLQJWRDVWHDGLQFUHDVHLQWKHHeHFWLYHFRPSOH[LWRIWKHPRGHO
+DOWLQJWUDLQLQJEHIRUHDPLQLPXPRIWKHWUDLQLQJHUURUKDVEHHQUHDFKHGWKHQUHSUHVHQWVDZDRI
OLPLWLQJWKHHeHFWLYHQHWZRUNFRPSOH[LW

,QWKHFDVHRIDTXDGUDWLFHUURUIXQFWLRQLHWKHUHJUHVVLRQSUREOHPZHFDQYHULIWKLVLQVLJKWDQGVKRZ
WKDWHDUOVWRSSLQJVKRXOGH[KLELWVLPLODUEHKDYLRXUWRUHJXODUL]DWLRQXVLQJDVLPSOHZHLJKWGHFDWHUP
7KLVFDQEHXQGHUVWRRGIURP)LJXUH,ILQWKHDEVHQFHRIZHLJKWGHFDWKHZHLJKWYHFWRUVWDUWVDW
WKHRULJLQDQGSURFHHGVGXULQJWUDLQLQJDORQJDSDWKWKDWIROORZVWKHORFDOQHJDWLYHJUDGLHQWYHFWRUWKHQ
WKHZHLJKWYHFWRUZLOOPRYHLQLWLDOOSDUDOOHOWRWKH D[LVWKURXJKDSRLQWFRUUHVSRQGLQJURXJKOWR
DQGWKHQPRYHWRZDUGVWKHPLQLPXPRIWKHHUURUIXQFWLRQ 6WRSSLQJDWDSRLQWQHDU LVWKHUHIRUH
VLPLODUWRZHLJKWGHFD7KHUHODWLRQVKLSEHWZHHQHDUOVWRSSLQJDQGZHLJKWGHFDFDQEHPDGH
TXDQWLWDWLYHWKHUHEVKRZLQJWKDWWKHTXDQWLW ZKHUH LVWKHLWHUDWLRQLQGH[DQG LVWKHOHDUQLQJ
UDWHSDUDPHWHUSODVWKHUROHRIWKHUHFLSURFDORIWKHUHJXODUL]DWLRQSDUDPHWHU 7KHHeHFWLYHQXPEHU
RISDUDPHWHUVLQWKHQHWZRUNWKHUHIRUHJURZVGXULQJWKHFRXUVHRIWUDLQLQJ

5HJXODULVDWLRQ

)LJXUH$VFKHPDWLFLOOXVWUDWLRQRIZKHDUOVWRSSLQJFDQJLYHVLPLODUUHVXOWVWRZHLJKWGHFDLQWKHFDVHRID
TXDGUDWLFHUURUIXQFWLRQ

$FWLYLW1HXUDO1HWZRUN

$FWLYLW1HXUDO1HWZRUN
,QWKLVDFWLYLWZHLPSOHPHQWDEDVLFODHUIHHGIRUZDUGQHXUDOQHWZRUNWRSUDFWLFHEDFNSURSDJDWLRQ
DOJRULWKP7KLVDFWLYLWKHOSVRXWRFRPSOHWH$VVLJQPHQW

3OHDVHGRZQORDGWKLVKWWSVZZZDOH[DQGULDUHSRVLWRURUJZSFRQWHQWXSORDGV$FWLYLW]LS]LSfOHULJKW
FOLFNDQGWKHQFKRRVHVDYHOLQNDVWKDWFRQWDLQVWKHGDWDVHWFVY5VFULSWU-XSWHUQRWHERRN
LSQEDQGRXWSXWfOHKWPO,IRXKDYHDFFHVVWRD-XSWHULQVWDQFHWKHQRWHERRNDQGGDWDVHWLV
HQRXJKWRUXQWKHH[SHULPHQWV2WKHUZLVHRXPDUHDGWKHLQVWUXFWLRQVDQGGLVFXVVLRQVIURPWKH+70/
fOHDQGH[HFXWHWKH5VFULSW

)RUGHWDLOHGGLVFXVVLRQDERXWVHWWLQJRXUSURJUDPPLQJHQYLURQPHQWXSSOHDVHUHIHUWR$SSHQGL[%
6HWWLQJXS