INFO3406 Project Stage 1 Explore, Clean, Define Overview The objective of stage 1 is to explore a data set and define a research question based on a research/business requirement. Activities include: (1) selecting a data set; (2) exploring, summarising and preparing the data; and (3) defining the problem and project requirements. Report (13 marks) The […]
data science 数据科学计算分析代写 INFO3406 Project Stage 1 Read More »
INF 553 – Spring 2018 Assignment 4 Community Detection Deadline: 04/09 2018 11:59 PM PST Assignment Overview In this assignment you are asked to implement the Girvan-Newman algorithm using the Spark Framework in order to detect communities in the graph. You will use only video_small_num.csv dataset in order to find users who have the similar
spark scala代写: INF 553 Assignment 4 Community Detection Read More »
INF 553 – Spring 2018 Assignment 5 Streaming Data Deadline: 04/23 2018 11:59 PM PST Assignment Overview In this assignment we’re going to implement some streaming algorithms. One is some analysis of Twitter stream. The other runs on a simulated data stream. Environment Requirements Python: 2.7 Scala: 2.11 Spark: 2.2.1 IMPORTANT: We will use these
spark scala代写: INF 553 Assignment 5 Streaming Data Read More »
Download and install Pig: cd wget http://rasinsrv07.cstcis.cti.depaul.edu/CSC555/pig-0.15.0.tar.gz gunzip pig-0.15.0.tar.gz tar xvf pig-0.15.0.tar set the environment variables (this can also be placed in ~/.bashrc to make it permanent) export PIG_HOME=/home/ec2-user/pig-0.15.0 export PATH=$PATH:$PIG_HOME/bin Use the same vehicles file. Copy the vehicles.csv file to the HDFS if it is not already there. Now run
In this section we are going to use Hive to run a few queries over the Hadoop framework. These instructions assume that you are starting from a working Hadoop installation. It should be sufficient to start your instance and the Hadoop framework on it. Hive commands are listed in Calibri bold font
CSC 555: Mining Big Data Project, Phase 2 (due Friday, March 16th) In this part of the project, you will various queries using Hive, Pig and Hadoop streaming. The schema is available below, but don’t forget to apply the correct delimiter: http://rasinsrv07.cstcis.cti.depaul.edu/CSC555/SSBM1/SSBM_schema_hive.sql The data is available at: http://rasinsrv07.cstcis.cti.depaul.edu/CSC553/data/ (this is Scale4) In your submission, please
大数据挖掘 hadoop spark 代写 CSC 555 Mining Big Data Project Phase 2 Read More »
CSC 555 Mining Big Data Assignment 5 Due Tuesday, 3/6 Suggested Reading: Hadoop: The Definitive Guide Ch19; Mining of Massive Datasets: Ch9 Solve 9.3.1-a, 9.3.1-e Where does Spark typically read the data from (and how does it ensure that data is not lost when a failure occurs)? What is the difference between content-based
大数据挖掘spark hadoop代写: CSC 555 Mining Big Data Assignment 5 Read More »
2017/18 COM6012 – Assignment 2 Assignment Brief Deadline: 11:59PM on Friday 27 April 2018 How and what to submit Create a .zip file containing two folders. One folder per exercise. Name the two folders: Exercise1, Exercise2. Within each folder, include the .sbt file, the .scala files, the .sh files, and the files you get as
机器学习spark scala代写: COM6012 – Assignment 2 Read More »
COMP9313 2018s1 Project 2 Problem statement – single target shortest path: Given a graph and a node “t”, find the shortest distances of all nodes to “t” together with the paths. For example, the shortest distance from node 1 to t is 7 with path 1->3->4->t. Please note that this is different from the single-
hadoop代写: COMP9313 Project 2 single target shortest path: Read More »
COMP9313 2018s1 Project 1 (10 marks) Problem statement: Given a directed graph, compute the average length of the in-coming edges for each node. Input files: In the input file, each line is in format of: “EdgeId FromNodeId ToNodeId Distance”. In the above example, the input is like: 0 0 1 10.0 1 0 2 5.0
hadoop代写: COMP9313 Project 1 Read More »