FIT3152 Data analytics: Assignment 1
This assignment is worth 20% of your final marks in FIT3152. Due: Friday 23rd April 2021.
Activity, language use and social interactions in an on-line community. Analyse the metadata and linguistic summary from a real on-line forum and submit a report of your findings. Do the following:
a.
• •
b.
• • •
c.
• •
d.
•
Analyse activity and language on the forum over time. Some starting points:
Describe your data: How active are participants, and are there periods where this increases or decreases? Is there a trend over time?
Looking at the linguistic variables, do these change over time? Is there a relationship between variables?
Analyse the language used by groups. Some starting points:
Threads indicate groups of participants communicating on the same topic. Describe the threads present in your data.
By analysing the linguistic variables for all or some of the threads, is it possible to see a difference in the language used by different groups?
Does the language used within threads (or between threads) change over time? How consistent or variable is the language used within threads?
Challenge: Social networks online. We can think of participants posting to the same thread at similar times (for example during the same month) as forming a social network. When these participants also post to other threads over the same period, their social network extends.
Can you define, graph and describe the social network that exists at a particular point in time, for example over one month? How does this change in the following months? Note: you only need to analyse a small portion of the social network over a short time period. We will cover social network analysis in Lecture 5.
Reflection on your investigation. What did you first investigate? How did you then modify your research based on the results of your first investigation?
Using one of the data science methodologies in Lecture 4, illustrate your research process.
The data is contained in the file webforum.csv and consists of the metadata and linguistic analysis of posts over the years 2002 to 2011. You will each work with 20,000 posts, randomly selected from the original file. The linguistic analysis was conducted using Linguistic Inquiry and Word Count (LIWC), which assesses the prevalence of certain thoughts, feelings and motivations by calculating the proportion of key words used in communication. See http://liwc.wpengine.com/ for more information, including the language manual http://liwc.wpengine.com/wp- content/uploads/2015/11/LIWC2015_LanguageManual.pdf
Create your individual data as follows:
rm(list = ls())
set.seed(XXXXXXXX) # XXXXXXXX = your student ID
webforum <- read.csv("webforum.csv")
webforum <- webforum [sample(nrow(webforum), 20000), ] # 20000 rows
Data
Data fields are (see the language manual for more detail and examples):
Column
Brief Descriptor
ThreadID
Unique ID for each thread
AuthorID
Unique ID for each author
Date
Date
Time
Time
WC
Word count of the text of the post
Analytic
LIWC Summary (Analytical thinking)
Clout
LIWC Summary (Power, force, impact)
Authentic
LIWC Summary (Using an authentic tone of voice)
Tone
LIWC Summary (Emotional tone)
WPS
LIWC (Words per sentence)
i
LIWC ("I, me, mine" words) First person singular
we
LIWC ("We, us, our" words) First person plural
you
LIWC ("You" words) Second person
they
LIWC ("They" words) Third person plural
number
LIWC(Quantities and ranks)
affect
LIWC (Expressing sentiment)
posemo
LIWC (Positive emotions)
negemo
LIWC (Negative emotions)
anx
LIWC (Indicating anxiety)
Submission. Due Friday 23rd April 2021 11:55pm GMT+10.
Suggested length: 6–8 A4 pages + appendix.
Submit the results of your analysis, answering the research questions and report anything else you discover of relevance. If you choose to analyse only a subset of your data, you should explain why.
You are expected to include at least one multivariate graphic summarising key results. You may also include simpler graphs and tables. Report any assumptions you’ve made in modelling, and include your R code as an appendix. Submit your report as a single PDF with the file name FirstnameSecondnameID.pdf on Moodle.
Software
It is expected that you will use R for your data analysis and graphics and tables. You are free to use any R packages you need but please document these in your report and include in your R code.
Assessment criteria will include:
The quality of your analysis and description of your analytical process; Graphics and tables supporting your analysis; The quality of graphics used in the report. Justification of your findings and the degree of proof you can offer (for example statistical tests); Readability and quality of your written report; Insights gained from the data; Novelty of your approach.
Factors you should consider (starting points, not a complete list):
Techniques: summary/descriptive statistics, identification of important variables, networks, etc. Major grouping variables: author, thread, date and/or time, or a combination of these.
Time window (days, weeks, months, years...); Subsets of the data to be analysed.
Graphics to communicate your analysis and insights (histograms, scatterplots, heat maps, time series are some basic starting points, but see https://datavizproject.com/ for inspiration.