COMP9313: Big Data Management Course web site: http://www.cse.unsw.edu.au/~cs9313/ Chapter 6.1: Mining Data Streams Data Streams ❖ In many data mining situations, we do not know the entire data set in advance ❖ Stream Management is important when the input rate is controlled externally: ➢ Google queries ➢ Twitter or Facebook status updates ❖ We can […]
代写代考 COMP9313: Big Data Management Read More »
COMP9313: Big Data Management Course web site: http://www.cse.unsw.edu.au/~cs9313/ Chapter 4.2: I Download and Configure Spark ❖ Current version: 3.1.2. https://spark.apache.org/downloads.html ➢ You also need to install Java first ❖ After downloading the package, unpack it and then configure the path variable in file ~/.bashrc export SPARK_HOME=/home/comp9313/workdir/spark export PATH=$SPARK_HOME/bin:$PATH ❖ Spark comes with four widely used
代写代考 COMP9313: Big Data Management Read More »
COMP9313: Big Data Management Course web site: http://www.cse.unsw.edu.au/~cs9313/ Part 1: X ❖ GraphX is Apache Spark’s API for graphs and graph-parallel computation. ❖ At a high level, GraphX extends the DD by introducing a new Graph abstraction: a directed multigraph with properties attached to each vertex and edge ❖ To support graph computation, GraphX exposes
代写代考 COMP9313: Big Data Management Read More »
COMP9313: Big Data Management Course web site: http://www.cse.unsw.edu.au/~cs9313/ Mining Data Streams II : Filtering Data Streams Filtering Data Streams ❖ Each element of data stream is a tuple ❖ Given a list of keys S ❖ Determine which tuples of stream are in S ❖ Obvious solution: Hash table ➢ But suppose we do not
代写代考 COMP9313: Big Data Management Read More »
COMP9313: Big Data Management Course web site: http://www.cse.unsw.edu.au/~cs9313/ Chapter 4.1: Part 1: ntroduction Limitations of MapReduce ❖ MapReduce greatly simplified big data analysis on large, unreliable clusters. It is great at one-pass computation. ❖ But as soon as it got popular, users wanted more: ➢ More complex, multi-pass analytics (e.g. ML, graph) ➢ More interactive
代写代考 COMP9313: Big Data Management Read More »
COMP9313: Big Data Management Course web site: http://www.cse.unsw.edu.au/~cs9313/ : MapReduce III Design Pattern 3: Order Inversion Computing Relative Frequencies ❖ “Relative” Co-occurrence matrix construction ➢ Similar problem as before, same matrix ➢ Instead of absolute counts, we take into consideration the fact that some words appear more frequently than others Word wi may co-occur
代写代考 COMP9313: Big Data Management Read More »
COMP9313: Big Data Management Course web site: http://www.cse.unsw.edu.au/~cs9313/ Chapter 1: Course Information and Introduction to Big Data Management Part 1: Course Information Course Info ❖ Lectures: 10:00 – 12:00 (Tuesday) and 14:00 – 16:00 (Thursday) ➢ Purely online (access through Moodle) ❖ Labs: Weeks 2-10 ❖ Consultation (Weeks 1-10): Questions regarding lectures, course materials, assignements,
代写代考 COMP9313: Big Data Management Read More »
COMP9313: Big Data Management Course web site: http://www.cse.unsw.edu.au/~cs9313/ Chapter 5.2: V Part 1: APIs A Brief Review of RDD ❖ The RDD is the most basic abstraction in Spark. There are three vital characteristics associated with an RDD: ➢ Dependencies (lineage) When necessary to reproduce results, Spark can recreate an RDD from the dependencies
代写代考 COMP9313: Big Data Management Read More »
COMP9313: Big Data Management Course web site: http://www.cse.unsw.edu.au/~cs9313/ Chapter 9: Recommender Systems Recommendations Recommendations Products, web sites, blogs, news items, … Recommender Systems Recommender Systems ❖ Application areas ➢ Movie recommendation (Netflix) ➢ Related product recommendation (Amazon) ➢ Web page ranking (Google) ➢ Social recommendation (Facebook) Netflix Movie Recommendation Why using Recommender Systems? ❖ Value
代写代考 COMP9313: Big Data Management Read More »
COMP9313: Big Data Management Course web site: http://www.cse.unsw.edu.au/~cs9313/ Chapter 7.2: Finding Similar Items Docu- ment Candidate pairs: those pairs of signatures that we need to test for similarity of strings of length k that appear in the doc- ument Signatures: short integer vectors that represent the sets, and reflect their similarity Step 3: Locality-Sensitive Hashing:
代写代考 COMP9313: Big Data Management Read More »