python network代写 CS3640, Assignment 1 Building a picture of the Internet

CS 3640: Introduction to Networks and Their Applications [Fall 2018]

Instructor: Rishab Nithyanand | Office hours: Wednesday 9-10 am or by appointment Teaching assistant: Md. Kowsar Hossain | Office hours: Monday 1:30-2:30 pm

Assignment 1: Crafting a picture of the Internet Released on: August 30th, 2018 | Due by: September 13th, 2018 (11:59:59 pm)

Maximum score: 100 | Value towards final grade: 13%

I. Groups

As stated in class, on the website, and on Piazza, this assignment will be done in groups of 4 or 5. Here are the groups for this assignment.

Group ID

1 2 3 4 5 6 7 8 9

10 11 12 13

Hawk IDs of group members

‘kdzhou’, ‘okueter’, ‘lburden’, ‘tnlowry’, ‘xxing2’ ‘mcagley’, ‘yzheng19’, ‘tsimonson’, ‘mfmrphy’ ‘dstutz’, ‘msmith3’, ‘sklemm’, ‘ymann’ ‘jmagri’, ‘zzhang103’, ‘uupadhyay’, ‘atran4’ ‘jglowacki’, ‘ppeterschmidt’, ‘kzhang24’, ‘tgoodmn’ ‘zhenbsong’, ‘bzhang22’, ‘ywang391’, ‘cweiske’ ‘xiaosong’, ‘bchoskins’, ‘jpflint’, ‘hpen’ ‘apatrck’, ‘xchen117’, ‘rdong6’, ‘weigui’ ‘jdhatch’, ‘susmerano’, ‘jpthiede’, ‘yitzhou’ ‘zluo1’, ‘godkin’, ‘nsonalkar’, ‘nicgoh’ ‘jstoltz’, ‘lye1’, ‘shangwchen’, ‘ywang455’ ‘hrunning’, ‘apizzimenti’, ‘yonghfan’, ‘jblue’ ‘gongyzhou’, ‘trjns’, ‘awestemeier’, ‘gmich’

Getting in touch with your team

Use the HawkIDs listed to email your team-mates or make a post on Piazza. If you’re unable to get in touch with any member of your team, let me know ASAP!

Congeniality and healthy collaboration

I will take complaints of discrimination and harassment extremely seriously and these will be reported to the appropriate authorities. You are expected to treat your classmates with respect and kindness.

II. Submission

Each group is to submit this notebook after filling in the blank cells as instructed. The submissions are due on ICON at 23:59:59 on September 13th, 2018. The last submission submitted by a team member before midnight on the due date will be the one graded unless ALL team members let the TA and me know that they want another submission to be graded (the late penalty if a submission made past the due date is chosen).

Late submissions

I am being generous in the amount of time allotted to this assignment to account for difficulties in scheduling meetings, etc. There will be no extensions of the due date under any circumstances. If a submission is received past the due date, the late policy detailed on the course webpage will apply.

Team-mate feedback

Each team member may also send me an email (rishab-nithyanand@uiowa.edu (mailto:rishab- nithyanand@uiowa.edu)) with subject “Feedback: Assignment 1, Group N” detailing their experience working with each of their team-mates. For each team member, tell me at least one good thing and one thing they could improve. These will be anonymized and released to each individual at the end of the term. It’s important to know how to work well in a team and early feedback before you move on to bigger and better things is always helpful. Sending feedback for all 4 assignments will fetch you a 4% bonus at the end of the term. Note: Sending with an incorrect subject line means that the email will not get forwarded to the right inbox.

III. Learning goals

My goal is to give you assignments that will help you build a nice starter pack of computer networking skills that will help you get a better picture of how the Internet works in addition to helping you in the real-world. Each of these tasks will be useful if you ever need to develop Internet app, manage a network, or do research. By the end of this assignment you should:

Know how to sniff packets leaving your computer (Task 1)

Here are some scenarios where I’ve needed to programmatically sniff packets on the network:

During Web and mobile app development. Being able to capture specific packets leaving your computer will allow you to inspect the packets generated by your app to see if they’re being created as expected. It helps with understanding if you’re getting errors because of something weird in the application/network layer of your code.

Identifying anomalous network events. Knowing how to sniff packets on the network will allow you to programmatically detect anomalies in network traffic which may be caused by a rogue end-host or a network attack. It can help you pin-point the victim and adversary causing the anomalous behaviour.

Know how to programmatically craft a packet in Python (Task 2)

Here are some scenarios where I’ve needed to craft a packet myself:

Identify badly configured or malicious routers and violations of the end-to-end principle.

Being able to craft your own packet will allow you to test if there are specific headers (control information) that cause a router to behave in an unexpected way — either because they are malicious or because they are poorly configured. We use this all the time in research to identify censors and proxies. If you want to know more, just ask me how.

Know how to programmatically trace the route taken by your packet and understand the functions of ICMP (Task 3)

Here are some scenarios where I’ve needed to know how my packets are being routed:

Identify pain-points in the network. Being able to observe the routers handling your traffic will allow you to figure out which routers in the network are experiencing heavy load. Knowing this will allow you to act before they fail — e.g., by installing a load-balancer.
Identify which entities are able to observe your Internet traffic. Besides just being something cool to know, this will come in handy if you ever decide to get involved in privacy and anonymity research.

IV. Team Information

Fill in your team information in this cell.
Group ID: XX
Group members: AAA, BBB, CCC, DDD, EEE

V. Task 1: Programmatically sniffing packets using Python and Scapy

Maximum score: 30 points

Step 1: 5 points Step 2: 20 points Step 3: 5 points

These are all the functions that should be sufficient for you to write a packet sniffer using Scapy. You do not have use just these, however. If you use other functions, do all the imports in this cell. You may not use functions not provided by Scapy.

You can read about what these functions do and generally find more Scapy documentation here:

https://scapy.readthedocs.io/en/latest/index.html (https://scapy.readthedocs.io/en/latest/index.html)

You may use the Internet to find more resources. Just make sure to cite them in the “credit reel cell” at the end of this notebook.

If you’re using a system on which Scapy is not already installed, you can install it using “pip install scapy” on the Linux terminal or the Anaconda command prompt on Windows. [some systems might require you to use “sudo”]

In [ ]:

In [ ]:

from scapy.all import sniff, ifaces

[5 points] Step 1: Print all the interfaces available on your device.

Background: An “interface” is basically the link between your computer and the network you are connected to. It generally operates at the link layer. Typically you should find an ethernet and a wireless interface interface listed in the output if you’re using a laptop.

Sample output:

INDEX  IFACE                                           IP
   MAC

14     Marvell AVASTAR Wireless-AC Network Controller  192.168.0.194
   C4:9D:ED:27:F1:CA

16     Surface Ethernet Adapter                        128.255.45.21
1  28:16:A8:07:DB:C1
8      Bluetooth Device (Personal Area Network)

   C4:9D:ED:27:F1:CB

[15 points] Step 2: Write a function that takes an interface, a list of host IPs, and a maximum packet count as input and sniffs packets leaving the input interface and prints all packets to/from any of the host IPs. The function should stop after max_count packets have been sniffed. Return the packets to/from any of the host IPs.

Hint: You will have to look at the source and destination IP address fields to match the host IP address. There are two versions of IP: IPv4 and IPv6. Most packets are IPv4, but you also will need to check source and destination fields of IPv6 packets.

TL;DR: Make sure to look at IPv6 headers in addition to IP headers.

In [ ]:

[10 points] Step 3
a: Call the function you just wrote and supply it with the interface being used by your computer,

max_count=10**3 , host_ips=[“2620:0:e50:6810::80ff:6044”, “128.255.96.68”] .

b: Open your browser and navigate to “http://homepage.divms.uiowa.edu/~rnithyanand/cs3640- f18/assignment-1/nggyu-ra.mp4″ (http://homepage.divms.uiowa.edu/~rnithyanand/cs3640- f18/assignment-1/nggyu-ra.mp4”) and then to “http://128.255.96.68/” (http://128.255.96.68/”) (in another tab) and then to “https://www.youtube.com” (https://www.youtube.com”) (in another tab). Enjoy the music for a while. If everything is working correctly, you should have a bunch of packets going to and from my UIowa homepage. Print the following information from the headers of each layer:

link layer: dst , src . [source and destination]
network layer: dst , src . [source and destination IP addresses] transport layer: dport , sport . [source and destination ports]

Sample output (a snippet from the full output):

In [ ]:

def find_packets_to_hosts(interface, host_ips, max_count):

Packet #40
Link layer: 28:16:a8:07:db:c1 (src link address) -> e4:c7:22:67:58:c
2 (dst link address)
Network layer: 2620:0:e50:1018:f083:c947:5041:a54c (src IPv6 addres
s) -> 2620:0:e50:6810::80ff:6044 (dst IPv6 address)
Transport layer: 55595 (src port) -> 80 (dst port)
Packet #41
Link layer: e4:c7:22:67:58:c2 (src link address) -> 28:16:a8:07:db:c
1 (dst link address)
Network layer: 128.255.96.68 (src IPv4 address) -> 128.255.45.211 (d
st IPv4 address)
Transport layer: 80 (src port) -> 55593 (dst port)

VI. Task 2: Programmatically crafting a packet using Scapy Maximum score: 30 points

Step 1: 5 points Step 2: 5 points

Step 3: 10 points Step 4: 10 points

These are all the functions that should be sufficient for you to craft a packet using Scapy. You do not have use just these, however. If you use other functions, do all the imports in this cell. You may not use functions not provided by Scapy.

You can read about what these functions do and generally find more Scapy documentation here:

https://scapy.readthedocs.io/en/latest/index.html (https://scapy.readthedocs.io/en/latest/index.html)

You may use the Internet to find more resources. Just make sure to cite them in the “credit reel cell” at the end of this notebook.

In [ ]:

from scapy.all import IP, ICMP, hexdump

[5 points] Step 1: From the set of packets you captured in Task 1, identify the first packet carrying application layer data (“HTTP”), used IPv4, and went from your computer to 128.255.96.68 . Print the entire contents of this packet — including all headers and application layer data. Save it in a variable named target_packet .

Sample output:

Packet #6
###[ Ethernet ]###

dst src type

###[ IP ]###
     version

= e4:c7:22:67:58:c2
= 28:16:a8:07:db:c1
= 0x800

=4
=5
= 0x0
= 456
= 17862
= DF
=0
= 128
= tcp
= 0x0
= 128.255.45.211 = 128.255.96.68

     ihl
     tos
     len
     id
     flags
     frag
     ttl
     proto
     chksum
     src
     dst
     \options   \
###[ TCP ]###

sport
dport
seq
ack
dataofs
reserved  = 0

= 55959
= http
= 3801435958L
= 1506251082
= 5

                    flags
                    window
                    chksum
                    urgptr
                    options

ip, deflate\r\nAccept-Language: en-US,en;q=0.9\r\n\r\n’ Hint: Application layer data is in the transport layer payload .

In [ ]:

                  = PA
                  = 2053
                  = 0x91d0
                  = 0
                  = []

                     = 'GET / HTTP/1.1\r\nHost: 128.255.96.68\r\nCon
nection: keep-alive\r\nCache-Control: max-age=0\r\nUpgrade-Insecure-
Requests: 1\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
 AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/
537.36\r\nDNT: 1\r\nAccept: text/html,application/xhtml+xml,applicat
ion/xml;q=0.9,image/webp,image/apng,*/*;q=0.8\r\nAccept-Encoding: gz

###[ Raw ]###
           load

[5 points] Step 2: Modify the application layer payload to show a different user-agent. The user- agent of the modified packet should be CS3640|Assignment-1|GroupID:NN (replace NN with your team ID). Print the contents of the modified packet.

Sample output:

0000  474554202F20485454502F312E310D0A GET / HTTP/1.1..
0010  486F73743A203132382E3235352E3936 Host: 128.255.96
0020  2E36380D0A436F6E6E656374696F6E3A .68..Connection:
0030  206B6565702D616C6976650D0A436163  keep-alive..Cac
0040  68652D436F6E74726F6C3A206D61782D he-Control: max-
0050  6167653D300D0A557067726164652D49 age=0..Upgrade-I
0060  6E7365637572652D5265717565737473 nsecure-Requests
0070  3A20310D0A557365722D4167656E743A : 1..User-Agent:
0080  204353333634307C41737369676E6D65  CS3640|Assignme
0090  6E742D317C47726F757049443A4E4E0D nt-1|GroupID:NN.
00a0  0A444E543A20310D0A4163636570743A .DNT: 1..Accept:
00b0  20746578742F68746D6C2C6170706C69  text/html,appli
00c0  636174696F6E2F7868746D6C2B786D6C cation/xhtml+xml
00d0  2C6170706C69636174696F6E2F786D6C ,application/xml
00e0  3B713D302E392C696D6167652F776562 ;q=0.9,image/web
00f0  702C696D6167652F61706E672C2A2F2A p,image/apng,*/*
0100  3B713D302E380D0A4163636570742D45 ;q=0.8..Accept-E
0110  6E636F64696E673A20677A69702C2064 ncoding: gzip, d
0120  65666C6174650D0A4163636570742D4C eflate..Accept-L
0130  616E67756167653A20656E2D55532C65 anguage: en-US,e

0140  6E3B713D302E390D0A0D0A
###[ Ethernet ]###

n;q=0.9….

dst src type

###[ IP ]###
     version

= e4:c7:22:67:58:c2
= 28:16:a8:07:db:c1
= 0x800

=4
=5
= 0x0
= 447
= 19401
= DF
=0
= 128
= tcp
= 0x0
= 128.255.45.211 = 128.255.96.68

     ihl
     tos
     len
     id
     flags
     frag
     ttl
     proto
     chksum
     src
     dst
     \options   \
###[ TCP ]###

sport
dport
seq
ack
dataofs
reserved  = 0

flags
window
chksum

= PA
= 2053
= 0x91c7

= 56643
= http
= 3015529313L
= 3280833796L
= 5

                    urgptr    = 0
                    options   = []

Hint: You will need to use “hexdump” to modify the application layer data. Look for more information here: https://scapy.readthedocs.io/en/latest/usage.html (https://scapy.readthedocs.io/en/latest/usage.html)

In [ ]:

[10 points] Step 3: Write a function that creates ICMP packets with the IP destination set to the address passed via an argument and a TTL value set to a value passed via another argument. Return the newly created icmp packet.

Background: ICMP (Internet Control Message Protocol) is a network layer protocol specifically built for the purposes of troubleshooting and logging. Think of it as an extension of the IP protocol. ICMP packets are encapsulated within an IP packet. The packet contains some information to aid with debugging. You can find more information about ICMP and its echo here: https://en.wikipedia.org/wiki/Internet_Control_Message_Protocol (https://en.wikipedia.org/wiki/Internet_Control_Message_Protocol) and https://tools.ietf.org/html/rfc792 (https://tools.ietf.org/html/rfc792)

Hint: You can learn how to compose packets utilizing different protocols in each layer of the stack here: https://scapy.readthedocs.io/en/latest/usage.html#stacking-layers (https://scapy.readthedocs.io/en/latest/usage.html#stacking-layers)

In [ ]:

def generate_icmp_packet(destination_ip, ttl):

[10 points] Step 4: Using the function you wrote in the previous step, generate three ICMP packets. The first one with a destination IP address of 128.255.96.68 (UIowa), the next with a destination IP address of 8.8.8.8 (Google), and the last with a destination IP address of 129.94.124.115 (UNSW – Sydney). Print the contents of these packets. Set the TTL to 100 for each of these packets.

Sample output (a snippet from the full output):

In [ ]:

google
###[ IP ]###

  version   = 4
  ihl       = None
  tos       = 0x0
  len       = None
  id        = 1
  flags     =
  frag      = 0
  ttl       = 100
  proto     = icmp
  chksum    = None
  src       = 128.255.45.211
  dst       = 8.8.8.8
  \options   \

###[ ICMP ]###
     type
     code

chksum id
seq

###[ Raw ]###
        load

= echo-request
= 0
= None
= 0x0

= 0x0
= ‘xxxx’

VII. Task 3: Programmatically trace the route taken by your packet Maximum score: 30 points

Step 1: 10 points Step 2: 5 points Step 3: 15 points

These are all the functions that should be sufficient for you to trace your packets using Scapy. You do not have use just these, however. If you use other functions, do all the imports in this cell. You may not use functions not provided by Scapy.

You can read about what these functions do and generally find more Scapy documentation here:

https://scapy.readthedocs.io/en/latest/index.html (https://scapy.readthedocs.io/en/latest/index.html)

You may use the Internet to find more resources. Just make sure to cite them in the “credit reel cell” at the end of this notebook.

In [ ]:

In [ ]:

In [ ]:

from scapy.all import sr

[10 points] Step 1: Write a function which sends packets passed to it and collects responses to these packets. Return the responses received.

Hint: Use sr() from Scapy, not sr1() . sr1() returns only the first response. Use the timeout argument of sr() to set the maximum time you’re willing to wait for a response. 10 is

usually a good value.

def send_and_receive_packets(packets_to_send):

[5 points] Step 2: Use the function you created in the previous step to send the three ICMP packets you crafted in task 2. Print the responses you get for each of them.

Sample output (a snippet from the full output):

Begin emission:
Finished sending 1 packets.

Received 2 packets, got 1 answers, remaining 0 packets
google    (<IP  frag=0 ttl=100 proto=icmp dst=8.8.8.8 |<ICMP  |<Raw

 load='xxxx' |>>>, <IP  version=4 ihl=5 tos=0x0 len=32 id=2849 flags
= frag=0 ttl=123 proto=icmp chksum=0x75da src=8.8.8.8 dst=128.255.4
5.211 options=[] |<ICMP  type=echo-reply code=0 chksum=0xf0f id=0x0

 seq=0x0 |<Raw  load='xxxx' |<Padding  load='\x00\x00\x00\x00\x00\x0
0\x00\x00\x00\x00\x00\x00\x00\x00' |>>>>)

[15 points] Step 3: Write a function which takes an IP address as an input argument and calls generate_icmp_packet() in a loop. In each iteration of the loop, the TTL argument passed to generate_icmp_packet() should be reduced by 1 (start at 10) and the destination_ip

argument passed should be set to the same IP address passed to the function you’re writing. Send these packets and record the responses using send_and_receive_packets() . Print the source IP address of the response for each TTL value. Call this function with the destination_ip set to

8.8.8.8 .

Basically, answer this question: Where did the response come from when I sent an ICMP packet with TTL=n?

In [ ]:

Hint: You may need to use try and except to handle timeouts and other errors that may arise from non-responsive routers.

Sample output (a snippet from the full output):

   TTL value: 10 | Source IP in response: 8.8.8.8
   TTL value: 9 | Source IP in response: 8.8.8.8
   TTL value: 8 | Source IP in response: 8.8.8.8
   TTL value: 7 | Source IP in response: 8.8.8.8
   TTL value: 6 | Source IP in response: 74.125.37.43
   TTL value: 5 | Source IP in response: 108.170.243.174
   TTL value: 4 | Source IP in response: 205.213.119.62
   TTL value: 2 | Source IP in response: 128.255.2.129
   TTL value: 1 | Source IP in response: 128.255.44.1

def get_all_hops(destination_ip):

The IP addresses you see in your output are from all the network devices (routers, load-balancers, etc.) which handled your packets every time you tried to communicate with 8.8.8.8 ! They give you an insight into how your packets traverse the Internet and the globe. You basically implemented the traceroute function! This is something that is used every day millions of times by people configuring their networks, doing research, etc.

Important: Be able to explain how traceroute works! You can read more about its working in Section 4.2 here: https://www2.eecs.berkeley.edu/Pubs/TechRpts/1997/CSD-97-945.pdf (https://www2.eecs.berkeley.edu/Pubs/TechRpts/1997/CSD-97-945.pdf)

You can use the code you just wrote anytime you want to see who gets to look at your packets later in the term when we’ll explore how to identify the owners of these IP addresses and their real-world locations.

Task 4: Credit reel!

You get 10 points just for submitting this assignment with a well written and formatted credit reel! Use the following cell to:

tell us what references you used to complete this assignment — e.g., online documentation, stackoverflow posts, etc.
who helped you IRL or on Piazza
which team-mates did which parts of the assignment

An example is filled in.

References used:

1. URL 1 to figure out how to do blah blah blah in Task 1.

2. URL 2 to figure out how to do blah blah blah in Task 2.

Who helped us:

1. XYZ helped us with the functions to use to do Task 3 (Piazza). 2. ABC helped us with the functions to use to do Task 3 (IRL).

Team credits:

1. AAA thought of the idea of blah blah for Task 2. She also wrote the complete implementation of Task 2, step 3.
2. BBB did the implementation of Task 3, step 1.