CIT 594 Solo Project
An Exercise in Software Design
Planning and design are key to successfully building complex software; these are the focus of this assignment. You will apply the design principles and design patterns that we recently covered in class to develop, from scratch, a Java application that reads text files as input and analyzes the contents.
This assignment is a significant level-up from previous assignments. You will not just be plugging code into an application structure that has been supplied to you. Instead, you will have to use the skills you have been learning all semester to design both the full structure of the application and the individual functions to perform its specific tasks. This is a much more time-consuming project than those you have encountered previously, so please get started early and plan your time accordingly.
Copyright By PowCoder代写 加微信 powcoder
Because of the centrality of design in this assignment, design will also be a large part of your grade. A perfectly organized submission that fails to compile will receive more points than a submission that works perfectly but violates all of the specified design objectives.
The good news is that much of the design work you do in this project will be directly reusable in the next assignment, your final group project. You must complete the current assignment by yourself, but doing your best work now will give you a head start when you begin to work with your assigned team.
1 Learning Ob jectives 3
2 Background 3
3 Input Data Format 3
3.1 Tweets:Tab-Separated…………………………….. 4 3.2 Tweets:JSON ………………………………… 4 3.3 States …………………………………….. 5 3.4 ProvidedFiles…………………………………. 6
4 Functional Specifications 6
4.1 Runtimearguments………………………………. 6 4.2 Identifyingrelevanttweets…………………………… 7 4.3 Determiningthelocationsoftweets………………………. 8 4.4 Programoutput………………………………… 9 4.5 Logging ……………………………………. 9 4.6 FunctionalityChecklist…………………………….. 10
5 Design Specification 11
5.1 TheLogger ………………………………….. 11
6 Logistics 13
6.1 Testing ……………………………………. 13
6.2 GettingHelpontheDiscussionBoard …………………….. 13 6.3 HowtoSubmit………………………………… 14 6.4 Assessment ………………………………….. 14 6.5 CommonMistakes&FrequentlyAskedQuestions……………….. 15
Learning Objectives
In completing this assignment, you will learn how to:
• Design a software system using an N-tier architecture
• Design software using the principles of modularity, functional independence, and abstraction • Apply the Singleton design pattern
• Use a Java library to read data stored in a JSON file
Background
Government agencies such as the Centers for Disease Control can use social media information to get an understanding of the spread of infectious disease. By analyzing the use of words and expressions that appear over time on platforms such as Twitter, Facebook, etc., it is possible to estimate where the disease is affecting people and whether it is spreading or appears to be contained.
In this assignment, you will design and develop an application that analyzes a small set of Twitter data to look for tweets regarding occurrences of the flu and determines the US states in which these tweets occur.
Information about the format of the input data, the functional specification of the application, and the way in which it should be designed follows below. Please be sure to read all the way through before you start programming!
Input Data Format
Your analytical dataset is a collection of potentially flu-related tweets, with some metadata for each. At a minimum, each tweet record will contain the tweet text, the tweet date, and the tweet location in latitude/longitude format. Additional metadata may also be present, but is not needed
for your analysis.
Some tweet records may appear in the dataset multiple times with identical data. You should analyze, count, and log each appearance independently, and not worry about trying to detect duplicate tweets.
Tweet records will be provided in two different formats: a tab-separated text file and a JSON file. Your program will need to automatically select the appropriate parser for a given file based on its type. You may infer the format of a file from its file name extension (the portion of the file name following the last “.”). Note that the provided “flu tweets.txt” and “flu tweets.json” files both contain the same set of tweets, just in different formats and with slightly different extraneous metadata.
Tweets: Tab-Separated
The tab-separated values file for this assignment has “.txt” as its file extension. Each line of the file contains the data for a single tweet. The following is an example of the data for a single tweet:
[41.38, -81.49] 6 2019-01-28 19:02:28 Yay, homework!
The line contains four tab-separated (“\t”) fields:
1. Location: the geographical coordinates [latitude, longitude] of the point of origin of the tweet. This field is demarcated by square brackets (“[]”) and the latitude and longitude are separated by a comma and a space (“, ”): “[41.38, -81.49]”
2. An identifier used by the collector of the tweets. (“6” in this example.) This field can be ignored for our purposes.
3. The date of the tweet in YYYY-MM-DD hh:mm:ss format. (“2019-01-28 19:02:28” in this example.) We will also ignore this field.
4. The text of the tweet: “Yay, homework!”
Tweets: JSON
JSON (“JavaScript Object Notation”) is a popular standard for exchanging data on the World Wide Web. For details see: ECMA-404 and rfc8259. In this assignment and elsewhere, JSON files use the “.json” extension. In brief, JSON files are human-readable text files which encode data in JavaScript syntax (which is also similar to Python syntax). Permissible atomic values are strings, numbers, or one of true, false, or null. All other data is encoded in one of two composite types: “object” and “array”.
JSON objects are effectively maps written in the same syntax as Python dicts: curly braces (“{}”) surrounding comma separated key:value pairs. Arrays are represented as square brackets (“[]”) enclosing a series of comma separated values, like Python lists. In general, JSON allows for arbitrary nesting of composite types.
The JSON tweets file contains an array of tweet objects. In JSON, the example tweet above might look something like:
“location”: [41.38, -81.49],
“identifier”: 6,
“time”: “2019-01-28 19:02:28”,
“text”: “Yay, homework!”
Note that if you open the provided JSON tweet archive in a text editor (which you should do to familiarize yourself with its structure), you may see that the JSON tweet objects contain additional fields beyond the ones that we are using, that certain unused fields are missing, or that the fields are in a different order from the example given above. This should not affect your work on this assignment, as you are not expected to attempt to manually parse the JSON from the file text. Instead, you will read in the files using a standard JSON processing library, and work with the resulting data structures.
There are numerous Java libraries for reading JSON objects, and numerous tutorials on how to use them. For this assignment, we’re going to be using the JSON.simple library. Use the provided json-simple-1.1.1.jar from the starter files; add it to your project’s build path. A tutorial for this library is available here. Do not put the jar file in your src or bin directories, and do not unpack it. Jars are meant to be used directly.
To repeat: Do not attempt to write your own code to parse the JSON file! It would be extremely time-consuming to get all the details right, and would take you far afield from the focus of this assignment. Only process the JSON file using the provided JSON.simple library.
In order to determine the state from which each tweet originated, your program will also need to read a file that contains the latitude and longitude of the center of each of the 50 US states, plus Washington DC, in comma-separated format. Each line of the file contains the data for a single state and contains the name of the state, the latitude, and the longitude. Here is an example:
Alabama,32.7396323,-86.8434593
Provided Files
Your program will be evaluated using the following input files:
• a set of 10,000 tweets in tab-separated format (flu tweets.txt)
• the same 10,000 tweets in JSON format (flu tweets.json)
• a CSV file listing the centers of the 50 U.S. states and Washington, D.C. (states.csv)
Download the three input files along with json-simple-1.1.1.jar and add them to your project’s root directory so that you can test your program. Identical copies of those files will be used as part of the functional evaluation of your submission. You should, of course, create your own input files for testing.
Functional Specifications
This section describes the specification that your program must follow. Some parts may be under- specified; you are free to interpret those parts any way you like, within reason, but you should ask a member of the instruction staff if you feel that something needs to be clarified. Your program must be written in Java. As with previous assignments, you should use Java 11 for this project since this is the level used by Codio. Do not configure a module for your project (even if your IDE recommends doing so). It’s possible your IDE might generate a module-info.java file even without prompting you; we recommend deleting this.
Runtime arguments
The runtime arguments to the program should specify, in this order:
• the name of the tweets input file
• the name of the states input file
• the name of the log file (for logging debug information; see “Logging” below)
For example: flu tweets.json states.csv log.txt
Do not prompt the user for this information! These should be specified when the program is started (e.g. from the command line or using a Run Configuration in your IDE). If you do not know how to do this, please see the documentation here.
The program should display an error message and immediately terminate upon any of the following 6
conditions:
• the number of arguments is incorrect
• the tweets file does not match (case-insensitive) a recognized extension (“.json” or “.txt”)
• the specified tweets file or states file does not exist or cannot be opened for reading (e.g.
because of file permissions); take a look at the documentation for the java.io.File class if you
don’t know how to determine this
• your program cannot create/open the specified log file for writing
For simplicity, you may assume that the tweets file and states file are well-formed according to the specified formats, assuming they exist and can be opened. You can also assume that the format is correctly labeled if the file name includes a valid extension.
These are pretty big assumptions but will greatly simplify this assignment!
Note: If the designated log file does not exist, it should be created, and if it does exist, it should be opened in append mode instead of overwriting the existing file. (Consult the FileWriter docs to get the parameters correct.)
Identifying relevant tweets
The goal of our tweet analysis is to track spread of the flu. To that end, you must identify the tweets that discuss the disease. After identifying the flu tweets, your program should match the locations of these tweets to states and print out the number of flu tweets in each state (but only for those states that had any flu tweets at all).
A tweet is considered to be a flu tweet if the text contains one or more flu words or hashtags. For this assignment, we’ll simplify the notion of a hashtag to the same thing as a word with a “#” in front (i.e. “#flu” is hashtag and “flu” is not). A valid flu word or hashtag satisfies the following criteria:
• The word must start with “flu” (or “#flu” for a hashtag). None of “influence”, “influenza”, and “!flu” are flu words, for our purposes.
• If there are any characters following “flu”, the next character must not be a letter, but any other following string does not invalidate the word. “fluent” is not a valid flu word, but“flu2020” is, as is “flu4me&u”.
• Matching should be case-insensitive, as in our sensitivity analysis earlier in the semester. “flu”, “FLU”, “Flu”, and “fLu” are all valid flu words.
• A flu hashtag is a flu word with a single ‘#’ in front of it.
Hint: Skim the regular expression documentation in java.util.regex.Pattern. There are meth-
ods in Pattern and Matcher that can make this quite easy if used correctly. Some example tweet classifications, note this does not cover all cases to consider:
I feel like I have the flu and I hate it
Flu symptoms are the worst
I definitely have the flu
I think I have the #flu I’m so sick
How would I know if I have the flu?
Five days I’ve had the flu! so sad
That bunny is so fluffy I wanna squeeze it Don’t be influenced by fake news
ugh, I got #fluvid-19, shoulda taken the vax. so sick with the #flue gonna go home now
Flu tweet?
Yes Yes Yes Yes Yes Yes No No No No
Although a real-world implementation would arguably want to identify the last example as a flu tweet, you should not do so for the purposes of this assignment.
Please do not spend too much time worrying about what is and what is not a “flu tweet”, as that is not the main purpose of the assignment. We’re not trying to trick you, we promise! As long as your code works correctly for the examples provided above, and is case-insensitive, you’ll be fine.
As you’ve surely noticed, this approach to identifying “flu tweets” is extremely simplified and probably not super-accurate, but determining whether a tweet really does indicate that someone has the flu is waaaay outside the scope of this assignment. So even if you believe you have a more accurate solution to the problem, for your submission, please be sure to follow the specifications described here!
Determining the locations of tweets
Once you have found the “flu tweets,” you will need to determine the state in which each originated.
For the purpose of this assignment we will use a simplified approach to matching locations. The state of origin is defined as the state whose provided reference point (from “states.csv”) is the lowest cartesian (planar) distance from the tagged location of the tweet. In the event of an exact tie, pick one arbitrarily. All alternatives in such a scenario will be considered equally valid.
Clearly this is not a perfect measure of distance — the Earth is a spheroid rather than a plane (hopefully we can all agree on that), states are weird shapes rather than single points, and we might
be interested in other completely different definitions of distance (e.g. travel time), or distances to other things like cities, or general clustering. If you find yourself inspired to look into these or other more accurate and meaningful metrics, that’s great! But your program will be graded on how well it matches the expected results, which are calculated using “flat-earth” cartesian distances in the form:
distance = (longitude2 − longitude1)2 + (latitude2 − latitude1)2 Program output
When your program finishes looking for “flu tweets” and determining their locations, it should print the number of “flu tweets” per state to the console, using System.out, with the state names listed in alphabetical order. (Hint: think about which data structure you can use to make this a bit easier!)
When writing this summary output to the screen, please format it as one line per state. Each line should consist only of the state name followed by a colon (“:”) and one space, then the number of “flu tweets” from that state.
Alabama: 1
McMurdo Station: 4
Mons Olympus: 1
Pennsylvania: 1000
Tranquility Base: 2
If a state has no flu tweets, it should be omitted from the output list.
Please do not post questions in the discussion forum asking whether your state tally or tallies are correct! It is up to each student to determine the correct output of the program.
In addition to displaying the output as described above, your program must also record all of the “flu tweets” it identifies in the log file that was specified as an argument to the program.
For each “flu tweet”, write one line to the log file. The line must start with the state name followed by 1 tab (“\t”) followed by the text of the tweet. A good practice is to send each “flu tweet” to the logger as a separate logging request. This request should be sent as soon as you know both that it is a flu tweet and what state it is from.
For example:
Alabama I have the flu!
Note: only the requested information should be sent to the logger. Do not add more information. Additional details:
• Only flu tweets should be logged.
• Flu tweets should be logged in the order they were read (i.e., in the order they appear in the
• Each flu tweet should be logged exactly once.
• Duplicate entries in the input should be treated as distinct tweets. If a particular tweet
appears twice in the input, it should appear twice in the output.
• Your logs should contain no extra output or extraneous characters.
• If a log file with the same name already exists when the program starts, new log entries should
be appended to the end of the existing file, rather than overwriting it.
Functionality Checklist
Things to double check:
• Command line arguments: tweets file, states file, log file
• Support json and tab-separated tweets files
• Correct filtering for flu tweets
• Correct location matching between tweets and states
• Log handles both new and existing files
• Log flu tweets in the order they are read
• No extra duplication or omission of tweets (in the logs and the tallies)
• Prints state tallies in alphabetical order
• Only print states with tweets
• Formatting is correct (improperly formatted output will not count, even for partial credit)
• No extraneous output to System.out or to the log files
• No translation or alteration of the input text (i.e. do not trim, do not apply character set
translations)
• Do not use System.exit
Design Specification
In addition to satisfying the functional specification described above, your program must also use some of the architecture and design patterns discussed in the videos.
In particular, you must use the N-tier architecture to identify and then separate your application into functionally independent modules.
To help with the organization of your code (and to help with the grading), please adhere to the following conventions:
• the program’s “main” function must be in a class called Main, which should be in the edu.upenn.cit594 package
• the classes in the Presentation/User Interface tier should be in the edu.upenn.cit594.ui pack- age
• the classes in the Processor tier should be in the edu.upenn.cit594.processor package
• the classes in the Data Management (file/backend data input/output, except for logging) tier
should be in the edu.upenn.cit594.datamanagement package
• the class should be in the edu.upenn.cit594.logging package
• the classes you create to share data between tiers should be in the edu.upenn.cit594.util
Your Main class should be responsible for reading the runtime arguments (described above), creating all the objects, arranging the dependencies between modules, etc. Beyond these tasks, make sure Main doesn’t perform any duties that belong in other tiers. See the “Monolith vs. Modularity” reading assignment in Module 10 for an example if you are unsure how to do this.
Because your program must be able to read the set of tweets from either a tab-separated file or from a JSON file, you must design your application so that you have two classes to read the tweets input file: one that reads from a tab-sep
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com