COMP9321:
Data services engineering
Week 7: Introduction to Data Analytics
Term 1, 2021
By Mortada Al-Banna, CSE UNSW
2
Data Driven Organizations
3
Data Driven Organizations and Data Analytics
4
Data Driven Organizations and Data Analytics
Data Driven Organizations and Data Analytics
5
6
Data Used for Analytics
7
Data Used for Analytics
Click to add text
People Vs Machines
8
Types of Analytics
• Descriptive Analytics tells us what happened in the past and helps a business understand how it is performing by providing context to help stakeholders interpret information.
• Diagnostic Analytics takes descriptive data a step further and helps you understand why something happened in the past.
• Predictive Analytics predicts what is most likely to happen in the future and provides companies with actionable insights based on the information.
• Prescriptive Analytics provides recommendations regarding actions that will take advantage of the predictions and guide the possible actions toward a solution.
9
http://vis.berkeley.edu/papers/CrowdAnalytics/
Crowdsourcing Data Analytics
Click to add text
10
What is Machine Learning?
•
Machine learning is an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed.
Machine learning focuses on the development of “computer programs that can access data and use it to learn for themselves”.
•
Useful Terminology
11 https://www.slideshare.net/rahuldausa/introduction-to-machine-learning-38791937
12
https://towardsdatascience.com/machine-learning-probability-statistics-f830f8c09326
Useful Basic Statistics
• Mean: The average of the dataset.
• Median: The middle value of an ordered dataset.
• Mode: The most frequent value in the dataset. If the data have multiple values that occurred the most frequently, we have a multimodal distribution.
• Probability: is the measure of the likelihood that an event will occur in a Random Experiment.
• Bayes’ Theorem: describes the probability of an event based on prior knowledge of conditions that might be related to the event.
• Range: The difference between the highest and lowest value in the dataset.
13
Useful Basic Statistics
• Variance: The average squared difference of the values from the mean to measure how spread out a set of data is relative to mean.
• Standard Deviation: The standard difference between each data point and the mean and the square root of variance.
• Causality: Relationship between two events where one event is affected by the other.
• Covariance: A quantitative measure of the joint variability between two or more variables.
• Correlation: Measure the relationship between two variables and ranges from -1 to 1, the normalized version of covariance.
Machine Learning for Data Analytics
14
https://www.datasciencecentral.com/profiles/blogs/decoding-machine-learning-methods
15
Machine Learning for Data Analytics
1. Prepare your Data
2. Define and Initialize a Model
3. Train your Model (using your training dataset)
4. Validate the Model (by prediction using your test dataset)
5. Use it: Explore or Deploy as a web service 6. Update and Revalidate
Example of a General Flow
16 https://www.slideshare.net/rahuldausa/introduction-to-machine-learning-38791937
What is an Apple?
17 https://www.slideshare.net/rahuldausa/introduction-to-machine-learning-38791937
18
Machine Learning Methods
19
https://www.kdnuggets.com/2017/12/top-data-science-machine-learning-methods.html
Machine Learning Methods
20
Questions Machine Learning Can Answer
Questions Machine Learning Can Answer
21
Questions Machine Learning Can Answer
22
Questions Machine Learning Can Answer
23
24
Questions Machine Learning Can Answer
Regression Analysis
Supervised ML
Regression
Simple Linear Multiple Linear Polynomial Linear Regression Regression Regression
25 https://www.slideshare.net/Simplilearn/linear-regression-analysis-linear-regression-in-python-machine-learning-algorithms- simplilearn?qid=c0cbd932-3ad1-45ec-b123-36805981982d&v=&b=&from_search=4
How Linear Regression Works
26 https://hackernoon.com/supervised-machine-learning-linear-regression-in-python-541a5d8141ce
Linear Regression Example
Training Set
27 https://hackernoon.com/supervised-machine-learning-linear-regression-in-python-541a5d8141ce
Linear Regression Example
28 https://hackernoon.com/supervised-machine-learning-linear-regression-in-python-541a5d8141ce
Linear Regression Example
29 https://hackernoon.com/supervised-machine-learning-linear-regression-in-python-541a5d8141ce
Classification
• SupervisedLearning
• You need the data labelled with the correct answer to train the algorithm
• Trained classifiers then can map input data to a category.
30 https://www.slideshare.net/brianjlange/machine-learning-in-5-minutes-classification
Classification
31 https://www.slideshare.net/brianjlange/machine-learning-in-5-minutes-classification
Classification
32 https://www.slideshare.net/brianjlange/machine-learning-in-5-minutes-classification
Classification
33 https://www.slideshare.net/brianjlange/machine-learning-in-5-minutes-classification
Clustering
• Unsupervised Learning
• Automated grouping of objects into so called clusters
• Objects of the same group are similar
• Different groups are dissimilar
34 https://www.slideshare.net/brianjlange/machine-learning-in-5-minutes-classification
Clustering
35 https://https://towardsdatascience.com/clustering-based-unsupervised-learning-8d705298ae51
Clustering
36 https://www.slideshare.net/brianjlange/machine-learning-in-5-minutes-classification
37
Popular Machine Learning Tools
Popular Machine Learning Tools
• TensorFlow • scikit-learn
• PredictionIO
38
Further Reading and Useful Resources
• Book: Mastering Machine Learning with Scikit-Learn, Second Edition. Gavin Hackeling
• https://jakevdp.github.io/PythonDataScienceHandboo k/05.02-introducing-scikit-learn.html
• https://towardsdatascience.com/machine-learning-an- introduction-23b84d51e6d0
• https://towardsdatascience.com/machine-learning- probability-statistics-f830f8c09326
• https://www.digitalocean.com/community/tutorials/an- introduction-to-machine-learning
• http://gael-varoquaux.info/scikit-learn-tutorial/ 39
40
Q&A