程序代写代做代考 C cache html DNA 7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
Master
Worker
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 1/27
stnetnoC fo elbaT .selpmaxe dna dnuorgkcab rof lairotut stekcos dna sdaerht ,sessecorp nohtyP eht ot refeR
.revres ecudeRpaM ruoy tset ot esu nac uoy taht smargorp ecuder/pam elpmas lareves dedivorp evah eW .revres ecudeRpaM eht rehtar tub ,smargorp ecuder pam etirw ton lliw uoY
.retsam eht yb nevig snoitcurtsni no desab sksat )gnipuorg( gnitros ro ecuder ,pam gnimrofrep ,sdnammoc tiawa neht dna ,retsam eht htiw sevlesmeht retsiger seludom .stluaf eldnah dna ,srekrow eht tsgnoma krow etubirtsid ,sboj eht eganam ,sboj ecudeRpaM rof netsil lliw hcihw , eht :tcejorp siht ni seludom yramirp owt era erehT
!etorw uoy noitatnemelpmi ecudeRpaM a gnisu ,enihcam ruoy no boj ecudeRpaM yna nur ot elba eb lliw uoy ,tcejorp siht detelpmoc evah uoy ecnO .yrotcerid nevig a ot boj eht fo tuptuo eht etirw dna ,yaw eht gnola seruliaf gnildnah ,noitelpmoc ot boj hcae nur lliw tI .sboj ecudeRpaM dettimbus-resu etucexe lliw taht revres dedaerht-itlum ,ssecorp-itlum ,enihcam elgnis a eb lliw sihT .nohtyP ni revres ecudeRpaM a tnemelpmi lliw uoy ,tcejorp siht nI
.eerht ot owt fo spuorg ni detelpmoc eb ot tcejorp puorg a si sihT .0202 ,7 tsuguA TSE mp9511 :euD
gnidaeR rehtruF gnidarg dna gnittimbuS gnitseT elpmaxe hguorht-klaW noitacificepS revreS ecudeRpaM revreS ecudeRpaM eht nuR puteS
ecudeR paM :4 tcejorP 584 SCEE
noitcudortnI
1.3 noisreV :02S rof esaeleR laitinI golegnahC
ecuderpam-4p

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
$ pwd
/Users/awdeorio/src/eecs485/p4-mapreduce
$ pwd
/Users/awdeorio/src/eecs485/p4-mapreduce
$ git status
On branch master
Your branch is up-to-date with ‘origin/master’.
nothing to commit, working tree clean
$ git remote -v
origin https://gitlab.eecs.umich.edu/awdeorio/p4-mapreduce.git (fetch)
origin https://gitlab.eecs.umich.edu/awdeorio/p4-mapreduce.git (push)
.gitignore
$ pwd
/Users/awdeorio/src/eecs485/p4-mapreduce
$ head .gitignore
This is a sample .gitignore file that’s useful for EECS 485 projects. …
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 2/27
.lairotuT tnemnorivnE lautriV nohtyP 1 tcejorP eht gnisu tnemnorivne lautriv nohtyP a etaerC tnemnorivne lautriv nohtyP
.)snoitcurtsni( elif a evah dluohs uoY
.yrotisoper baLtiG etomer a ot detcennoc eb dluohs yrotisoper lacol ruoy dna sutats ”naelc“ a htiw yrotisoper lacol a evah dluohs uoy ,enod erʼuoy retfA
.lairotut maet a rof lortnoc noisreV eht ta kool dnoces a ekat osla thgim uoY .lairotut lortnoc noisreV eht gnisu lortnoc noisrev pu teS
.tnereffid eb thgim noitacol redlof ruoY .)snoitcurtsni( tcejorp siht rof redlof a etaerC redlof tcejorP
.redargotuA eht no puorg ruoy retsigeR noitartsiger puorG
lortnoc noisreV
puteS

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
source
env/bin/activate
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 3/27
$ pwd
/Users/awdeorio/src/eecs485/p4-mapreduce
$ ls -d env
env
$ echo $VIRTUAL_ENV
/Users/awdeorio/src/eecs485/p4-mapreduce/env
$ pwd
/Users/awdeorio/src/eecs485/p4-mapreduce
$ wget https://eecs485staff.github.io/p4-mapreduce/starter_files.tar.gz
$ tar -xvzf starter_files.tar.gz
starter_files/
$ pwd /Users/awdeorio/src/eecs485/p4-mapreduce
$ mv starter_files/* .
$ rm -rf starter_files starter_files.tar.gz
$ tree
.
├── mapreduce
│ ├── __init__.py
│ ├── master
│ │ ├── __init__.py
│ │ └── __main__.py
│ ├── submit.py
│ ├── utils.py
│ └── worker
│ ├── __init__.py
│ └── __main__.py
├── setup.py
├── tests
│ ├── correct
│ │ ├── grep_correct.txt
│ │ └── word_count_correct.txt
│ ├── exec
│ │ ├── grep_map.py
.llabrat dna yrotcerid lanigiro eht evomer dna yrotcerid tcejorp ruoy ot selif retrats eht evoM
.)
rebmemer( detavitca sʼti taht dna ,tnemnorivne lautriv nohtyP a evah uoy taht kcehC
.selif eseht ees dluohs uoY
.selif retrats eht kcapnu dna daolnwoD selif retratS

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 4/27
source env/bin/activate
pip install -e . # install the correct dep
VERSION
mapreduce
mapreduce/master/
mapreduce/worker/
mapreduce/submit.py
mapreduce/utils.py
setup.py
tests/
tests/exec/
tests/correct/
tests/input/
tests/input_small/
testdata/
│ │
│ │
│ │
│ │
│ │
│ ├── input
│ │ ├── file01
├── grep_reduce.py
├── wc_map.sh
├── wc_map_slow.sh
├── wc_reduce.sh
└── wc_reduce_slow.sh
…
│ │ └── file08
│ ├── input_small
│ │ ├── file01
│ │ └── file02
│ ├── testdata
…
│ ├── test_worker_8.py
│ └── utils.py
└── VERSION
stset cilbup ruo yb desu seliF gnitset tsaf rof tupni margorp ecuderpam elpmaS tupni margorp ecuderpam elpmaS tuptuo tcerroc margorp ecuderpam elpmaS tuodts dna nidts esu lla ,smargorp ecuderpam elpmaS stset tinu cilbuP noitarugifnoc egakcap nohtyP ecuderpam rekrow dna retsam neewteb derahs edoC boj ecudeRpaM wen a timbus ot edoc dedivorP siht tnemelpmi ,eludom noteleks rekrow ecuderpam siht tnemelpmi ,eludom noteleks retsam ecuderpam selif noteleks egakcap nohtyP ecuderpam selif retrats eht fo noisreV
.selif retrats eht fo hcae fo noitpircsed feirb a sʼereH
.piP htiw segakcap llatsni dna tnemnorivnE lautriV eht etavitca ot erus ekaM

mapreduce worker
mapreduce-
mapreduce-master
master
worker
mapreduce-submit
&
$ mapreduce-master 6000 &
$ mapreduce-worker 6000 6001 &
$ mapreduce-worker 6000 6002 &
pgrep -lf pgrep -af
$ pgrep -lf mapreduce-worker
15364 /usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/Res
15365 /usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/Res
$ pgrep -lf mapreduce-master
15353 /usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/Res
7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 5/27
.smetsys xuniL/UNG no dna XSO no esu :etoN .dnuorgkcab eht ni gninnur sessecorp ruoy eeS
.dnuorgkcab eht ni ssecorp eht trats ot snaem ) ( dnasrepma ehT .no netsil ot trop hcihw meht llet neht dna ,0006 trop no retsam eht htiw etacinummoc dluohs yeht taht meht llet dna ,srekrow owt pu trats ew ,nehT .PCT gnisu 0006 trop no netsil lliw hcihw retsam a pu trats lliw sihT .yletarapes pu meht trats ot evah lliw uoy os ,sessecorp etarepes sa nur rekrow eht dna retsam ehT .gnihtyna od tʼnow tsuj ti ,xob eht fo tuo nur lliw edoc retrats ehT
srekrow dna retsam a tratS .retsam eht ot boj wen a dnes ot dedivorp osla evʼeW .
htiw rekrow a dna a hcnuaL .seludom dna
tniop yrtne enil dnammoc eht htiw retsam sedulcni egakcap nohtyP a etirw lliw uoY
revres ecudeRpaM eht nuR edoc gniggol elpmas dedivorp osla evʼeW
ytilicaf gniggoL nohtyP )shtap metsyselif detneiro-tcejbO( yrarbiL bilhtaP nohtyP yrarbiL NOSJ nohtyP stekcoS nohtyP gnidaerhtitluM nohtyP ssecorpbuS nohtyP
.noitatnemelpmi rotcurtsni ruo ni desu ew seirarbil eht ot sknil kciuq emos era ereH .lairotut stekcos dna sdaerht ,sessecorp nohtyP eht etelpmoC seirarbiL
.yrotisoper tiG ruoy ot timmoc a ekam ot aedi doog a sʼti ,selif retrats naelc eht ot segnahc yna gnikam erofeB

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
$ pkill -f mapreduce-master
$ pkill -f mapreduce-worker
$ pgrep -lf mapreduce-worker # no output, because no processes $ pgrep -lf mapreduce-master # no output, because no processes
mapreduce/submit.py
$ mapreduce-submit –help
Usage: mapreduce-submit [OPTIONS]
Top level command line interface.
Options:
-p, –port INTEGER Master port number, default = 6000
-i, –input DIRECTORY Input directory, default=tests/input
-o, –output DIRECTORY Output directory, default=output
-m, –mapper FILE
-r, –reducer FILE
–nmappers INTEGER
–nreducers INTEGER
–help
Mapper executable, default=tests/exec/wc_map.sh
Reducer executable,
default=tests/exec/wc_reduce.sh
Number of mappers, default=4
Number of reducers, default=1
Show this message and exit.
$ pgrep -f mapreduce-master # check if you already started it $ pgrep -f mapreduce-worker # check if you already started it $ mapreduce-master 6000 &
$ mapreduce-worker 6000 6001 &
$ mapreduce-worker 6000 6002 &
$ mapreduce-submit –mapper tests/exec/wc_map.sh –reducer tests/exec/wc_reduce.sh
bin/mapreduce
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 6/27
.)lairotuT( secitcarp tseb tpircs llehs eht wollof ot erus eB
.selpmaxe gniwollof eht ni segassem eht tnirP . deman tpircs llehs a eb dluohs tI .sessecorp rekrow dna retsam ecuder pam eht no kcehc dna pots ,trats ot tpircs tini na etirw llʼeW .dnuorgkcab eht ni snur taht margorp a ,)nomead ro( ecivres a fo elpmaxe na si revres ecudeRpaM ehT
.detnemelpmi ton era rekroW dna retsaM esuaceb liaf ot boj eht tcepxe ew won thgiR .tpircs llehs a gnisu revres eht gnitrats yfilpmis llʼew ,retaL .boj a nur ot woh sʼereH
.stnemugra enil dnammoc gnisu boj eht yficeps nac uoY .tekcos PCT niam sʼretsaM eht ot boj a sdnes tI . dedivorp osla evah ew ,yltsaL
boj ecudeRpaM a timbuS
tpircs tinI
.sessecorp ruoy potS

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
mapreduce-master 6000 &
sleep 2
mapreduce-worker 6000 6001 &
mapreduce-worker 6000 6002 &
$ ./bin/mapreduce start
starting mapreduce …
$ ./bin/mapreduce start
Error: mapreduce-master is already running
|| true
nc
# Detect GNU vs BSD netcat. We need netcat to close the connection after # sending a message, which requires different options.
set +o pipefail # Avoid erroneous failures due to grep returning non-zero if nc -h 2>&1 | grep -q “\-c”; then
NC=”nc -c”
elif nc -h 2>&1 | grep -q “\-N”; then
NC=”nc -N”
elif nc -h 2>&1 | grep -q “\-C”; then
NC=”nc -C” else
echo “Error detecting netcat version.”
exit 1 fi
set -o pipefail
echo ‘{“message_type”: “shutdown”}’ | $NC localhost 6000 || true sleep 2 # give the master time to receive signal and send to workers
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 7/27
.retsam eht rof si elpmaxe gniwollof ehT .sessecorp rekrow dna retsam eht llik ,nehT
.) ( tacteN rof noitpo tcerroc eht tuo erugif yllacitamotua ew taht eciton oslA .ylrae tixe ot tpircs eht gnisuac morf egassem nwodtuhs ”ecin“ deliaf a tneverp lliw taht ecitoN .sdnammoc gniwollof eht etucexE
.gninnur ydaerla sʼti nehw revres trats yllatnedicca :elpmaxE
.sdnammoc gniwollof eht etucexe ,esiwrehtO .gninnur ydaerla si rekrow ro retsam a fi 1 tixE revres tratS
revres potS
elpmaxE

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
$ ./bin/mapreduce stop
stopping mapreduce …
./bin/mapreduce stop
stopping mapreduce …
killing mapreduce master …
killing mapreduce worker …
$ ./bin/mapreduce start
starting mapreduce …
$ ./bin/mapreduce status
master running
workers running
$ ./bin/mapreduce stop
stopping mapreduce …
killing mapreduce master …
killing mapreduce worker …
$ ./bin/mapreduce status
master not running
workers not running
$ ./bin/mapreduce restart
stopping mapreduce …
killing mapreduce master …
killing mapreduce worker …
starting mapreduce …
echo “killing mapreduce master …” pkill -f mapreduce-master || true
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 8/27
noitacificeps revres ecudeRpaM
.dellik si ssecorp dna egassem nwodtuhs ot dnopser tʼnseod revres ,2 elpmaxE
.egassem nwodtuhs ot sdnopser revres ,1 elpmaxE
elpmaxE revres tratseR
elpmaxE sutats revreS

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
handle_msg
mapreduce/master
mapreduce/worker/__main__.py
mapreduce/
port_number
*”
utils.py
mapreduce/worker
mapreduce/master/__main__.py
tmp
tmp
tmp
“job-
port_number – 1
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 9/27
) ( no netsil dluohs sihT .srekrow eht morf segassem taebtraeh PDU rof netsil lliw hcihw ,daerht wen a etaerC
esu dna bolg bilhtaP ees :TNIH . ni sredlof boj ecuderpam dlo yna eteleD
.htap elif a fo strap tnereffid ”rehtegot eulg“ ot rotarepo hsals bilhtaP eht esu :tniH .ridkm bilhtaP :tniH .ti peek ,stsixe ydaerla fI .revres ecudeRpaM eht yb desu selif etaidemretni lla erots lliw ew erehw si sihT . redlof wen a etaerC
:gniwollof eht od dluohs retsaM eht ,putrats nO .no netsil dluohs retsaM eht taht trop PCT yramirp ehT :
.tnemugra enil dnammoc eno ylno tpecca dluohs retsaM ehT weivrevo retsaM
.uoy ot pu yleritne si sessalc eht fo noitatnemelpmi lautca eht tub ,rekroW eht dna retsaM eht rof sceps noitacinummoc eht enifed ylno
lliw eW .retsaM dna rekroW htob ot nommoc edoc esuoh ot esu nac uoy hcihw
edisni a dedivorp osla evah eW .egakcap hcae edisni tif ees uoy sa selif
wen etaerc yam uoy ,segakcap nohtyP gnisu era ew ecniS .)
dna ni detrats uoy tog ew( sessalc owt eht enifed lliw
uoy erehw ,segakcap dna eht edisni og lliw edoc ruoY noitazinagro edoC
.ytilanoitcnuf nevig eht eldnah dluohs ssalc hcihw yfitnedi ot woleb snoitcesbus eht erofeb ]rekroW/retsaM[ tup eW
.NOSJ otni ti esrap tsum daerht ruoy tub sgnirts eviecer stekcos ;NOSJ gnisu dettamrof sgnirts eb lliw tcejorp siht ni noitacinummoc llA :etoN .noitcennoc PCT wen a gnisu detacinummoc eb dluohs egassem PCT hcaE .segassem wen rof pool elihw etinifni na ni gninetsil eunitnoc lliw retsaM eht ,noitucexe sdne dna snruter egassem eht nehW . dellac noitcnuf a ni deldnah si egassem yreve yas sʼtel ,ceps siht nI .tekcos PCT eht no netsil ot gniunitnoc erofeb noitelpmoc ot egassem taht eldnah dluohs ti ,tekcos PCT niam sti no egassem a seviecer rekroW/retsaM eht sa noos sA
.woleb detsil segassem eht naht rehto noitacinummoc yna no yler ton dluohs uoY .woleb detsil ew segassem eht rof gnikcehc eb ylno lliw lliw ew ,revres ruoy tset oT :etoN .segassem gnimocni lla rof stekcos PDU/PCT no gninetsil era srekroW/retsaM eht taht rebmemeR .)sessalc owt eht neewteb seicnedneped ro atad erom yna dda ton od .e.i( yltnednepedni krow dluohs rekroW eht dna retsaM eht dna ,woleb snoitacificeps tcaxe ruo wollof tsum uoY .uoy ot pu yleritne si noitatnemelpmi eht ,ceps noitacinummoc eht dna ytilanoitcnuf eht gninifed ylno era ew taht si trap nuf ehT .revres ecudeRpaM eht fo ytilanoitcnuf eht ebircsed ew ,ereH
.

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
listen()
port_number
listen()
try:
msg = json.loads(msg)
except JSONDecodeError:
continue
master_port
port_number
worker_port
listen()
register
register_ack
worker_port
listen()
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 10/27
:tekcos PCT niam eht no deviecer eb lliw dna mrof gniwollof eht fo eb lliw egassem nwodtuhs ehT .nwodtuhs revres etaitini ot egassem laiceps a eviecer nac retsaM ehT .tsrif detnemelpmi eb dluohs ti ,ylreporp noitcnuf ot nwodtuhs eriuqer stset ruo fo fo lla esuaceB
.retsaM eht htiw sretsiger yllufsseccus rekroW taht erofeb rekroW a morf segassem taebtraeh yna erongi ylefas dluohs retsaM ehT :ETON
.retsaM eht ot segassem taebtraeh gnidnes rof elbisnopser eb lliw hcihw daerht wen a etaerc ,egassem eht gniviecer nopU
.gnidoced NOSJ liaf taht esoht gnidulcni ,segassem dilavni erongI .rekrow eht fo emitefil elohw eht rof nepo niamer dluohs daerht eno ylno :etoN .noitcnuf eht llac dna nevig eht no tekcos PCT wen a etaerC
.retsam eht htiw retsiger ot esu neht dluohs ti hcihw ,DI euqinu sʼrekroW eht eb lliw sihT .rekroW eht fo DI ssecorp eht teG
:retsaM eht sa snoitca fo ecneuqes ralimis a od dluohs rekroW hcae ,noitazilaitini nO retsam eht
morf snoitcurtsni eviecer ot no netsil dluohs rekrow siht taht tekcos PCT ehT :
)rotcurtsnoc retsaM eht ni eht sa emas( no gninetsil ylevitca si retsaM eht taht tekcos PCT ehT :
]rekrow + retsam[ nwodtuhS
retsaM eht ot egassem eht dneS
.stnemugra enil dnammoc owt tpecca dluohs rekroW ehT weivrevo rekroW
.detixe evah sdaerht retsam lla litnu rotcurtsnoc retsam eht morf nruter ot tiaW
woleb nwohs sa egassem eht daol ot yrt ot uoy nehw tpecxe/yrt a esu segassem eseht erongi oT .gnidoced NOSJ liaf taht esoht gnidulcni ,segassem dilavni erongI !segassem gnimocni rof tiaW
.retsam eht fo emitefil elohw eht rof nepo niamer dluohs daerht eno ylno :etoN .noitcnuf eht llac dna nevig eht no tekcos PCT wen a etaerC
.lufpleh eb dluoc ecnarelot tluaf rof daerht rehtonA .deen yam uoy kniht uoy putes ro sdaerht lanoitidda yna etaerC

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
test_master_0
test_worker_0
test_integration_0
ready
busy
dead
{
“message_type” : “register”, “worker_host” : string, “worker_port” : int, “worker_pid” : int
}
{
“message_type”: “register_ack”, “worker_host”: string, “worker_port”: int, “worker_pid” : int
}
{
“message_type”: “shutdown”
}
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 11/27
.retal siht no eroM .staebtraeh gnidnes trats dluohs rekroW eht ,deviecer neeb sah egassem siht retfA .siht ekil dettamrof ,deretsiger sah rekroW eht gnigdelwonkca egassem a htiw dnopser neht lliw retsaM ehT
retsaM eht ot siht ekil egassem a dnes dluohs ti ,snoitcurtsni rof netsil ot ydaer si rekroW a ecnO .srekroW morf segassem noitartsiger rof netsil tsum retsaM ehT
emit fo tnuoma emos rof gnip ot deliaf sah rekroW : ksat a gnimrofrep si rekroW :
krow tpecca ot ydaer si rekroW :
:setats gniwollof eht ni eb nac srekroW .srekroW ydaer eht gnoma detubirtsid ylno si krow eht taht os emit nevig yna ta srekroW lla fo kcart peek dluohs retsaM ehT
]rekrow + retsam[ noitartsiger rekroW
.tsrif noitartsiger rekrow tnemelpmi ot deen llʼuoy tub , si tset nwodtuhs rehtonA .yletelpmoc dna ,trap tsrif sʼ ssap ot elba eb dluohs uoy ,tniop siht tA
.flesti etanimret dluohs retsaM eht ,srekroW lla ot egassem eht gnidrawrof retfA
.daerht elgnis a edisni neppah eseht htob sa egassem nwodtuhs eht eldnah ot elba gnieb erofeb ksat taht etelpmoc ot ti rof yako si ti ,)woleb debircsed sa( ksat a gnitucexe fo elddim eht ni ydaerla si rekroW eht fI .elbissop sa noos sa etanimret dluohs ,egassem nwodtuhs eht gniviecer nopu ,srekroW ehT .ti htiw deretsiger evah taht srekroW gnivil eht fo lla ot egassem siht drawrof dluohs retsaM ehT

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
test_integration_0
test_master_0 test_worker_1
{
“message_type”: “new_master_job”, “input_directory”: string, “output_directory”: string, “mapper_executable”: string, “reducer_executable”: string, “num_mappers” : int, “num_reducers” : int
}
tmp/job-{id}
tmp job-0/
mapper-output/
grouper-output/
reducer-output/
job-1/
mapper-output/
grouper-output/
reducer-output/
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 12/27
.noitucexe boj nigeb nac retsaM eht naht ,ysub ton si revres eht dna srekrow era ereht fI .noitucexe noitcnuf eht dne dna )txen debircsed( eueuq lanretni na ot dedda eb dluohs boj eht ,srekroW elbaliava on era ereht ro ,ysub si revres eht fI .boj a gnitucexe yltnerruc si revres ecudeRpaM eht rehtehw kcehc dna krow ot ydaer srekroW yna era ereht fi kcehc dluohs retsaM eht ,putes era seirotcerid eht retfA .egatS gnipuorG eht od ot etarepooc ot evah lliw srekroW dna retsaM eht tub ,yltnednepedni selif elbatucexe nevig eht gnisu gnicuder dna gnippam eht od lliw srekroW .gnicudeR ,gnipuorG ,gnippaM :segats 3 ni srucco boj ecudeRpaM hcae ,rebmemeR
:)boj hcae rof sredlof wen 4 etaerc dluohs uoy(elpmaxe siht elbmeser lliw erutcurts yrotcerid ehT .)sretnuoc lla ekil tsuj 0 ta gnitrats( retnuoc boj tnerruc eht si di erehw , mrof eht fo ,og lliw boj eht rof selif yraropmet eht fo lla erehw seirotcerid wen fo tes a etaerc lliw retsaM eht ,tseuqer boj a ot esnopser nI
:tekcos PCT niam sti no egassem gniwollof eht eviecer lliw retsaM eht ,boj wen a fo tneve eht nI ]retsam[ tseuqer boj weN
.
dna dna ssap ot elba eb dluohs uoy ,tniop siht tA
.sksat yna rekroW eht ngissa ot ksat tnerruc etelpmoc eht fo noitelpmoc litnu tiaw ro egats txen eht litnu tiaw nac ti ,ecuder/puorg/pam a gnitucexe ydaerla si retsaM eht fI .)deretsiger srekroW yna erofeb retsaM eht ta devirra evah dluoc boj a esuaceb( rekroW eht ot ngissa nac ti krow yna sah ti fi )retal debircsed( eueuq boj eht kcehc dluohs retsaM eht ,retsaM eht htiw sretsiger rekroW tsrif eht retfA

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
{
“message_type”: “new_worker_job”, “input_files”: [list of strings], “executable”: string, “output_directory”: string “worker_pid”: int
}
handle_msg
output_directory
test_master_1
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 13/27
ti fi elif etirwrevo( elif tupni eht sa emas eht eb dluohs seman elif tuptuo ehT .elif cificeps a ot dettuptuo si dna ni dradnats hguorht elbatucexe eht ot dessap si tupni ehT .)elif tupni
hcae no elbatucexe eht nur dluohs uoy dna elif tupni rep elif tuptuo eno(
nevig eht ot tuptuo eht gnitcerid elihw ,elif tupni deificeps eht revo elbatucexe nevig eht fo noitucexe trats lliw sti ,egassem ksat wen siht seviecer rekrow a nehW
.emit emas eht ta sksat elpitlum eldnah ot elba eb dluohs rekroW a ,rekroW a ot sksat erom sngissa ti erofeb egassem enod a rof tiaw ot deen ton seod retsam ehT .rekrow hcae ot sksat pam 2 dnes dluow ti ,esac siht nI .srekrow eht lla gnoma sksat eseht ecnalab ot tpmetta neht dluow tI .selif tupni 2 htiw 1 dna hcae elif tupni eno htiw 3 ,sksat 4 etaerc dluohs retsam ehT .deificeps sksat pam 4 dna selif tupni 5 ,elbaliava srekroW 2 era ereht erehw esac eht redisnoC
:od ot krow evah yeht taht wonk meht gnittel ,)tekcos PCT cificeps sʼrekroW hcae no( rekroW hcae ot mrof gniwollof eht fo egassem NOSJ a dnes lliw retsaM ehT .sksat hcus ynam ro ,eno ,orez teg dluoc rekroW hcaE .rof elbisnopser si ti krow tahw wonk rekroW hcae tel ot sdeen retsaM eht ,tupni eht gninoititrap retfA .)boj gnimocni eht ni deificeps sksat pam fo rebmun eht si ʼX‘ erehw( strap ʼX‘ ni selif tupni eht noititrap dna yrotcerid tupni eht nacs dluohs retsaM eht ,egatS gnippaM eht ffo trats oT
.sboj gnidnep rof eueuq boj eht kcehc dluohs uoy ,krow ot retsiger rekroW tsrif eht ees uoy nehw ,reilrae deton sA
.emit yna ta ksat ecudeRpaM eno etucexe ylno lliw revres ecudeRpaM ruoy ,tcejorp siht ni ,yticilpmis roF .egatS gnippaM sti gnitrats yb )srekroW ydaer era ereht fi dna( eno si ereht fi boj gnidnep txen eht ssecorp dluohs retsaM eht ,sehsinif boj a sa noos sA .nigeb nac egatS gnippaM sʼboj txen eht erofeb etelpmoc eb tsum sksat ecuder dna ,puorg ,pam sʼboj tnerruc eht taht snaem siht taht etoN .dehsinif sah eno tnerruc eht litnu eueuq lanretni na ni boj eht erots dna ,seirotcerid eht etaerc ,boj eht tpecca dluohs ti ,srekrow ydaer on erew ereht nehw ro eno gnitucexe ydaerla si ti elihw boj wen a seviecer retsaM a fI
]srekrow[ gnippaM
]retsam[ gninoititrap tupnI
]retsam[ eueuq boJ . ssap ot elba eb dluohs uoy ,tniop siht tA

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
output_directory
tmp/job-{id}/mapper-output
tmp/job-0/mapper-output/file01.txt
data/input/file01.txt
{
“message_type”: “status”, “output_files” : [list of strings], “status”: “finished”, “worker_pid”: int
}
test_worker_3 test_worker_4 test_worker_5
sorted01 sorted02
{
“message_type”: “new_sort_job”, “input_files”: [list of strings], “output_file”: string, “worker_pid”: int
}
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 14/27
:siht ekil kool dluohs srekroW eht ot tnes segassem ehT .cte … , :noitnevnoc gniman gniwollof eht esU .elif regral eno otni selif fo rebmun emos gnitros rof elbisnopser eb lliw rekroW hcaE
.egats siht gnirud eldi tis srekroW emos fi yako si ti ,srekroW naht selif rewef era ereht fI .meht gnoma ylneve selif eht ecnalab ot tpmetta dluohs retsaM eht ,srekroW naht selif erom era ereht fI .yek yb ton enil yb neppah dluohs egatS gnipuorG eht ni gnitroS .selif eht egrem dna tros ot srekroW sngissa dna ,sreppam eht yb detaerc selif eht fo lla ta skool retsaM eht ,egatS gnipuorG eht trats oT
rehtegot syek ekil gnipuorg ,tuptuo reppam detros segnarraer retsaM . tuptuo reppam hcae tros srekroW .
:spets owt sah egatS gnipuorG ehT .egatS gnipuorG eht trats lliw retsaM eht ,hsinif sreppam lla retfA
]srekrow + retsam[ gnipuorG . , , ssap ot elba eb dluohs uoy ,tniop siht tA
:mrof eht fo tekcos niam sʼretsaM eht ot egassem PCT a dnes dluohs ti ,ksat sti dehsinif sah rekroW a ecnO .elif tupni hcae rof yrotcerid tuptuo eht ot gnipip dna ,eno yb eno selif tupni eht revo elbatucexe deificeps eht gninnur rof elbisnopser si rekroW eht ,noitarepo fo epyt eht fo sseldrageR .sksat ecuder ro pam ot citsonga eb dluohs rekroW ehT
.noitcnuf )(nur.ssecorpbus yrarbil dradnats nohtyP eht eeS :tniH elif tuptuo
eht dna si elif tupni eht yficeps dluohs retsaM eht ,elpmaxe roF
.) .e.i( redlof tuptuo -reppam eht eb syawla lliw egatS gnippaM eht ni ehT .)stsixe ydaerla

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
{
“message_type”: “status”, “output_file” : string, “status”: “finished” “worker_pid”: int
}
XX heapq
reduceX
X
reduce01
reduce02 reduce03 reduce04
test_master_8
output_directory
test_master_2
reducer-output
{
“message_type”: “status”, “output_files” : [list of strings], “status”: “finished”
“worker_pid”: int
}
reducer-output
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 15/27
eht yb deificeps eulav ehT( egassem noitaerc boj lanigiro eht yb deificeps yrotcerid tuptuo lanif eht ot yrotcerid eht morf selif tuptuo eht evom dluohs retsaM eht ,boj nevig a rof sksat ecuder eht rof egassem ”dehsinif“ tsal eht deviecer sah retsam eht sa noos sA
:mrof eht fo tekcos niam sʼretsaM eht ot egassem PCT a dnes dluohs ti ,ksat sti dehsinif sah rekroW a ecnO
.elif tupni eht sa eman elif tuptuo emas eht esu ,niagA .redlof eht eb syawla lliw egatS gnicudeR eht ni ehT .elbatucexe ecuder ro pam tcerroc eht nur ot rekroW eht sllet ti erus gnikam rof elbisnopser si retsaM eht – nur ot dlot si ti elbatucexe eht snur tsuj rekroW ehT .ksat ecuder ro pam a gninnur si ti fi wonk ot deen tʼnseod ti – egatS gnippaM eht sa emas eht si siht ,rekroW eht oT
.gnipuorg nibor dnuor tneiciffe seifirev hcihw ,
ssap osla dluohs uoY .sesahp ecuder eht dna ,gnipuorg ,gnippam rof segassem tcerroc eht dnes nac retsam ruoy taht stset sihT . ssap ot elba eb dluohs uoy ,tniop siht tA
.yrotcerid tuptuo repuorg eht ni , ,
, etaerc dluohs retsam eht ,deificeps sksat ecuder 4 era ereht fI .rebmun ksat ecuder eht si erehw , deman eb dluohs eliF .egatS gnicudeR eht rof stupni eht eb lliw hcihw ,retsaM egatS gnipuorG eht fo tuptuo eht rof noitnevnoc gniman gniwollof eht esU
.elif detros egral elgnis a etaerc ton oD .ecno ta yromem otni tif nac tes atad eritne eht emussa ton oD .yrarbil eht esu dluohs uoy dna ,stsil detros K gnigrem ot suogolana si melborp sihT :tniH .nibor dnuor gnisu selif ot syek etacollA .sksat ecuder fo rebmun eht tneserper erehw ,selif otni yek yb selif detros eht sezinagroer retsaM eht ,etelpmoc si gnitros retfA
:swollof sa dettamrof egassem a kcab dnes dluohs ti ,dehsinif sah rekroW eht ecnO
]retsam[ pu gnipparW
]srekrow[ gnicudeR

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
output_directory
outputfilex
x
outputfile01, outputfile02, outputfile03, outputfile04
{
“message_type”: “heartbeat”, “worker_pid”: int
}
test_master_3
test_master_4
test_worker_2
test_integration_1
test_integration_2
test_integration_3
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 16/27
,
. dna, ,
, , ssap ot elba eb dluohs uoy ,tniop siht tA
.deid sah ksat taht dengissa ylsuoiverp saw ohw rekroW eht taht wonk uoy litnu srekroW tnereffid ot ksat emas eht dnes tʼnod ,sdrow rehto nI .noitacilpud diova tub ,ycnerrucnoc ezimixam ot tpmetta dluohs retsaM ruoY
.knil siht ot refer esaelp ,siht no ofni erom roF .yranoitcid eht revo gnitareti elihw deifidom era syek nehw rorre na ni tluser nac tI .erutcurts atad yranoitcid nohtyP eht no stniartsnoc ot eud si sihT .serutcurts atad lanretni sʼretsaM eht morf ti evomer ton od tub ,daed sa rekroW deliaf eht kram dluohs uoY .rekroW rehtona ot ksat taht ngissa ot evah lliw retsaM eht ,ksat a gnitucexe si ti elihw seid rekroW a fI .srekroW elbaliava gnoma krow etubirtsid ylneve ot tpmetta dluohs retsaM eht )gnicuder ,gnipuorg ,gnippam( noitucexe eht fo tniop hcae tA .srekroW evil ot detubirtsider eb dluohs )ton ro detelpmoc( sksat esoht fo lla neht ,ti ot dengissa sksat eht lla gnitelpmoc erofeb seid rekroW a fI
:sdnoces 2 yreve tnes eb dluohs dna ,siht ekil kool dluohs segassem ehT .PDU aiv retsaM ot setadpu dnes ot daerht taebtraeh a evah lliw rekroW hcaE
.enihcam rekroW rehtona ot rof elbisnopser saw ti krow revetahw ngissa dna ,deid sah ti taht emussa dluohs uoy ,wor a ni sgnip 5 naht erom sessim rekroW a fI .siht rof etadommocca tsum retsaM ruoY .meht dnes uoy taht sksat hsinif ton yam dna emit yna ta eid nac srekroW
.
.eueuq boj eht ni yltnerruc eno tʼnsi ereht fi sboj rof gninetsil ot kcab og ro ,boj elbaliava txen eht rof eueuq boj eht kcehC .tsixe ydaerla tʼnseod ti fi yrotcerid tuptuo eht etaerC meht emaner dluohs retsam
.ereh elpmaxe etelpmoc a eeS elpmaxe hguorht-klaW
]rekrow + retsam[ staebtraeh dna ecnarelot tluaF
eht ,selif tuptuo lanif 4 era ereht fI .rebmun elif tuptuo lanif eht si erehw , demaner eb dluohs selif eht ,yrotcerid tuptuo lanif eht nI .)yek
gnitseT

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
correct.txt
output.txt
submit.py output
$ cat tests/input/* | tests/exec/wc_map.sh | sort | \ tests/exec/wc_reduce.sh > correct.txt
$ cat output/* | sort > output.txt $ diff output.txt correct.txt
-vs
$ pwd
/Users/awdeorio/src/eecs485/p4-mapreduce
$ pytest -vs
Control C
$ time mapreduce-master 6000 INFO:root:Starting master:6000 …
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 17/27
.
.elbarugifnoc si ti tub , si yb dedivorp yrotcerid tuptuo tluafed ehT ,elif eno otni tuptuo sʼrevres ecudeRpaM ruoy etanetacnoc ,nehT .
.emit laer eht ot ralimis si emit resu eht esuaceb stiaw-ysub noitulos siht taht llet nac eW .snoitpecxe ro srorre yna erongI .semit lareves – gnisserp yb ti llik ,sdnoces wef a retfA .srekrow yna tuohtiw retsam a emit ,gnitiaw ysub tceted oT
.gnitiaw-ysub fo noitanalpxe na rof lairotuT nohtyP ni stekcoS dna sdaerhT ,sessecorP eht eeS .yllacol melborp ecnamrofrep eht eciton ton thgim uoy os ,tnemnorivne redargotua detcirtser eht naht lufrewop erom hcum ylbaborp si potpal ruoY .tuoemit a ot eud redargotua eht no liaf dna enihcam tnempoleved ruoy no ssap yam stiaw-ysub taht noitulos A
.segassem gniggol sedulcni hcihw ,tuptuo wohs ot tsetyp ot galf eht ddA .stset tinu cilbup eht nuR
.yltcaxe ceps siht ni detsil esoht hctam tsum metsys ruoy ni tnes segassem eht dna ,ceps noitacinummoc eht sediseb ycnedneped rehto on evah dluohs edoc ruoY .evoba deificeps ew segassem eht rof ylno retsam dna rekrow ruoy hctaw lliw redargotua eht taht etoN
.)woleb noitces ees( tpircs llehs a ni nettirw noitulos tnuoc drow a htiw uoy dedivorp osla evah ew ,siht tset uoy pleh oT !nohtyP ni nettirw sboj ecuder dna pam gninnur ot su timil ton dluohs revres ruoy – egaugnal yna ni eb nac selbatucexe ecuder dna pam eht taht etoN
ni srewsna tcerroc eht etareneg ot edoc ecudeRpaM ruoy tuohtiw enil dnammoc eht ta selbatucexe eseht nuR .elpmaxe ecuder dna pam tnuoc drow elpmis a dedivorp evah eW
.ytilanoitcnuf sʼnoitulos ruoy gnitset no detrats teg uoy pleh dluohs ,evoba elpmaxe hguorhtklaw ehT .nwo ruoy no noitulos ecuderpam ruoy fo serutaef emos tset ot tpmetta dluohs uoy ,revewoH .4 tcejorp rof etius tset cilbup evisneherpmoc yltneced a dedivorp evah eW
gnitiaw ysub rof tseT

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 18/27
$ time mapreduce-master 6000 INFO:root:Starting master:6000 …
real 0m3.530s
user 0m0.275s
sys 0m0.036s
tests/exec/wc_map_slow.sh
tests/exec/wc_reduce_slow.sh sleep
$ grep -B1 sleep tests/exec/wc_*slow.sh tests/exec/wc_map_slow.sh-# Simulate a long running job tests/exec/wc_map_slow.sh:sleep 3
—
tests/exec/wc_reduce_slow.sh-# Simulate a long running job tests/exec/wc_reduce_slow.sh:sleep 3
$ pkill -f mapreduce- # Kill any stale Master or Worker $ mapreduce-master 6000 & # Start Master
$ mapreduce-worker 6000 6001 & # Start Worker 0
$ mapreduce-worker 6000 6002 & # Start Worker 1
real 0m4.475s
user 0m4.429s
sys 0m0.039s
$ mapreduce-submit \
–input tests/input_small \
–output output \
–mapper tests/exec/wc_map_slow.sh \
–reducer tests/exec/wc_reduce_slow.sh \
–nmappers 2 \
–nreducers 2
.gnitucexe nigeb ot sreppam eht rof tiaw dna boj ecudeRpaM wols yllanoitnetni na timbuS
.pu trats ot meht rof tiaW .srekroW owt dna retsaM ecudeRpaM eno trats ,tsriF
.emit peels eht esaercni ot tnaw yam uoY .ksat gninnur wols a etalumis ot stnemetats esu selbatucexe esehT .
dna ni boj ecudeRpaM wols yllanoitnetni na dedivorp evah eW
.boj ecudeRpaM wols yllanoitnetni na gninnur sʼti elihw rekrow a llik ot si aedi lareneg ehT .ecnarelot tluaf yfirev uoy pleh lliw noitces sihT
.emit laer eht ot derapmoc llams si emit resu eht taht ecitoN .tiaw ysub ton seod elpmaxe sihT
ecnarelot tluaf gnitseT

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
sleep
pycodestyle pydocstyle pylint
kill pgrep
$ pgrep -af mapreduce-worker # Linux
$ pgrep -lf mapreduce-worker # macOS
77811 /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/Re 77893 /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/Re $ kill 77811
$ pgrep -lf mapreduce-worker
77893 /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/Re
$ pgrep -af mapreduce-worker | head -n1 | awk ‘{print $1}’ | xargs kill # Linux $ pgrep -lf mapreduce-worker | head -n1 | awk ‘{print $1}’ | xargs kill # macOS [2]- Terminated: 15 mapreduce-worker 6000 6001
$ pylint –disable=no-value-for-parameter –disable=method-hidden mapreduce
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 19/27
pgrep
setup.py
disable-copyfile
—
.SOcam no ylno galf
eht edulcnI .gro.584scee//:sptth morf deknil si hcihw ,redargotua eht ot llabrat a timbuS
.
gnidarg dna gnittimbuS ni dedivorp si tahw morf edisa seicnedneped lanretxe yna esu ton yam uoY
.erutaef noitativni wen etaerc eht gnisu redargotua eht no puorg ruoy retsiger dluohs rebmem maet enO
:siht ekil tnilyp nuR . dna , ,
morf sgninraw ro srorre on niatnoc dluohs edoc nohtyP lla ,stcejorp suoiverp ni sA
.noitacinummoc PCT dna putrats rof emit meht evig ot sdnammoc neewteb ni sdnammoc gnidda ,tset siht tpircS :pit-orP
?eviecer rekroW dnoces eht dna tsrif eht dluohs sksat gnicuder dna gnitros ynam woH ?deviecer evah rekroW dnoces eht dluohs sksat gnippam ynam woH ?deviecer evah rekroW tsrif eht dluohs sksat gnippam ynam woH
:redisnoc ot rekroW dna retsaM ruoy morf segassem gniggol ot daeR .roivaheb tcerroc eht yfirev ,yllaniF
.rekrow eht dellik yllautca uoy taht kcehc ot niaga esu nac uoY .DIP sti no esu neht dna ,rekrow a fo DIP eht dnif ot esu ylkciuQ .boj ecuder ro pam a gnitucexe si ti elihw srekrow eht fo eno lliK
.enil eno htiw rekrow eno llik ot yaw a sʼereH
elyts edoC

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
StopIteration
StopIteration
socket.recv()
worker_message_generator
socket.recv()
Mock
master_message_generator
StopIteration
socket.recv()
$ tar \
–disable-copyfile \
–exclude ‘*__pycache__*’ \
–exclude ‘*tmp*’ \
-czvf submit.tar.gz \
setup.py \
bin \
mapreduce
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 20/27
.egassem nwodtuhs eht si hcihw ,egassem ekaf tsal eht retfa dellac ton si erus ekaM
.desiar si noitpecxe a ,)segassem ekaf denifed -erp fo tuo nur srotareneg egassem eht( nevig si egassem on tub dellac si
nehW .)sliated rof tset hcae ni dna
ees( egassem ekaf denifed-erp a dnes lliw stset eht ,dellac si
revenehW .snoitcnuf tekcos eht ekaf ot ssalc a gnisu era ew ,stset eht nI
noitaretIpotS#lmth.snoitpecxe/yrarbil/3/gro.nohtyp.scod//:sptth .desiar si noitpecxe a ,detsuahxe si elbareti eht nehW .)elbareti na( ecneuqes dedocdrah a morf seulav nruter lliw )rekroW ekaf ro retsaM ekaf( snoitcnuf tset kcom eht ot sllac detaepeR :A
?rorre si tahW :Q .rotcurtsnoc eht ni secnatsni rekroW/retsaM ruoy pu trats uoy erus ekaM :A
.gnihtyna gniod ton ot smees edoc ym ,stset eht nur I nehw tub ,yltcerroc sdnammoc htiw rekroW/retsaM ym trats nac I :Q
%04
enildaed eht retfa nur stset tinu neddiH stset tinu cilbuP
%06 eulaV
elbarevileD
repap ecudeRpaM lanigiro sʼelgooG
gnidaer rehtruF
.cirbur etamixorppa na si sihT cirbuR
QAF

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
Failed to close threads
Master Worker
Connection Refused
TypeError: Object of type PosixPath is not JSON serializable JSON pathlib.Path
class PathJSONEncoder(json.JSONEncoder): “””
Extended the Python JSON encoder to encode Pathlib objects.
Docs: https://docs.python.org/3/library/json.html
Usage:
>>> json.dumps({
“executable”: TEST_DIR/”exec/wc_map.sh”,
}, cls=PathJSONEncoder)
“””
def default(self, o):
“””Override base class method to include Path object serialization.””” if isinstance(o, pathlib.Path):
return str(o)
return super().default(o)
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 21/27
:ti od ot woh sʼereH .gnirts a ot tcejbo a trevnoc ot woh wonk ton seod yrarbil nohtyP eht tluafed yB :A
? teg I od yhW :Q
.desolc era sdaerht lla nehw tixe ylno dna margorp ruoy trats srotcurtsnoc eht erus ekaM .margorp ruoy nur ot srotcurtsnoc ssalc / eht llac ew ,stset eht nI :A
?rorre ro a gnitteg I ma yhW :Q

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 22/27

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 23/27

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 24/27

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 25/27

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 26/27

7/29/2020 EECS 485 Project 4: Map Reduce | p4-mapreduce
https://eecs485staff.github.io/p4-mapreduce/#shutdown-master–worker 27/27