Introduction Background
MapReduce cluster
Development
Example MapReduce job Worker registration
Copyright By PowCoder代写 加微信 powcoder
Job submission
Fault tolerance Conclusion
This site uses Just the Docs, a documentation theme for Jekyll.
Search CS 162 HW 5
Background / Code Code
©À©¤©¤ Cargo.lock
©À©¤©¤ Cargo.toml
©À©¤©¤ build.rs
©À©¤©¤ data
©¦ ©À©¤©¤ alphabet2
©¦ ©¦ ©¸©¤©¤ …
©¦ ©À©¤©¤ graph-edges-medium
©¦ ©¦ ©¸©¤©¤ …
©¦ ©¸©¤©¤ gutenberg
©¦ ©¸©¤©¤ …
©À©¤©¤ proto
©¦ ©À©¤©¤ coordinator.proto
©¦ ©¸©¤©¤ worker.proto
©À©¤©¤ rust-toolchain.toml
©¸©¤©¤ src
©À©¤©¤ app
©¦ ©À©¤©¤ grep.rs
©¦ ©À©¤©¤ mod.rs
©¦ ©À©¤©¤ vertex_degree.rs
©¦ ©¸©¤©¤ wc.rs
©À©¤©¤ autograder
©¦ ©¸©¤©¤ mod.rs
©À©¤©¤ bin
©¦ ©À©¤©¤ autograder.rs
©¦ ©À©¤©¤ client.rs
©¦ ©À©¤©¤ coordinator.rs
©¦ ©¸©¤©¤ worker.rs
©À©¤©¤ client.rs
©À©¤©¤ codec
©¦ ©¸©¤©¤ mod.rs
©À©¤©¤ coordinator
©¦ ©À©¤©¤ args.rs
©¦ ©¸©¤©¤ mod.rs
©À©¤©¤ lib.rs
©À©¤©¤ log.rs
©À©¤©¤ proto
©¦ ©À©¤©¤ coordinator.rs
©¦ ©¸©¤©¤ worker.rs
©À©¤©¤ task
©¦ ©¸©¤©¤ mod.rs
©À©¤©¤ tests
©¦ ©¸©¤©¤ mod.rs
©À©¤©¤ utils.rs
©¸©¤©¤ worker
©À©¤©¤ args.rs
©¸©¤©¤ mod.rs
Protocol buffers for the coordinator and worker gRPC servers. You will need to modify the coordinator.proto and worker.proto files to allow the coordinators and workers to
communicate necessary information with one another.
MapReduce applications such as word count and distributed grep. You may not modify any of the existing files in this folder, although you are free to test out your own MapReduce apps if you like.
Code for the binaries that are generated by cargo build . You may not modify the code in
this folder, but feel free to take a look at the client if you are curious how it is implemented.
Helper functions for the MapReduce client.
src/codec/
Helper functions for serialization, which will come in handy when storing and retrieving data. You may not modify the code in this folder as our tests depend on it. The doc comments describe how to use the code in this module.
src/coordinator/
Code for the MapReduce coordinator. The majority of your changes will be likely be made in the mod.rs file, but you are free to create additional files to organize your code as you see fit.
src/lib.rs
Helper functions, types, and constants that may come in useful for your implementation. You may not modify this files, but you are encouraged to take a look at their contents.
src/log.rs
Logging utilities to help you debug your code.
src/proto/
Autogenerated code for the coordinator and worker gRPC stubs. The coordinator and worker stub modules are included in the map_reduce crate structure in src/lib.rs here:
/// Automatically generated utilities for RPCs.
pub mod rpc {
/// Includes RPC definitions for the coordinator.
pub mod coordinator {
include!(“proto/coordinator.rs”);
/// Includes RPC definitions for the worker.
pub mod worker {
include!(“proto/worker.rs”);
You can thus import their contents with crate::rpc::coordinator and crate::rpc::worker , respectively.
Module for encapsulating logic related to task and job management. You are not required to use it, but it may help keep your code clean.
Local MapReduce tests. We provide you with one basic test case, but you are free to add your own here. These tests are not run by the autograder.
src/utils.rs
Miscellaneous helper functions.
src/worker/
Code for MapReduce workers. Most of your changes will be made in the mod.rs file,
though you can add files here as needed.
Back to top
Data files for testing your MapReduce implementation. The alphabet2 and gutenberg folders contain data that can be used with the wc (word count) and grep applications, while the graph-edges-medium folder contains data for the vertex-degree application.
Copyright ý 2022 CS 162 staff.
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com