DCF255
DCF255
Lecture 2 | Data Communication Standards
Agenda
TCP/IP Protocol Architecture
TCP/IP Protocol in Brief
Application Layer and Protocol Specifications
Message Encoding – UTF-8
Standard Organizations
May require more than one slide
2
TCP/IP
Protocol Architecture
3
Late 1970’s
In the early days of networking high degree of dissatisfaction among business owners, venders and users
Different network vendor’s used different data formats and data exchange protocols
Had to buy software and hardware from same vendor.
“Vender lock-in” resulted in higher prices.
ISO created task force to develop Open System Inter-Connection (OSI).
TCP/IP was well established when the task force begin, but it was expected that the OSI would replace it
OSI
…we can expect to see continued usage of other open products such as the TCP/IP protocol suite (Transmission Control Protocol/Internet Protocol), which provides the basic needs of networking. Use of the official OSI protocols in data communications and networking will continue to gain popularity, and the OSI will continue to be utilized both in the development of OSI protocols and as a gauge for comparison with other protocols.
Gerald Cole, Computer Networking For System Programmers, John Wiley & Sons, New York, 1990, p.24
TCP/IP Remains Dominant – 5 Layer Protocol Stack
Development of World Wide Web created exponential growth of the Internet – 1991
Advances in network speed and reliability made the OSI 7 layer model seem unnecessarily complex
OSI 7 layer model used to teach networking, but not used in data communications
Except, physical layer and data link layers have nearly 100% compliance worldwide
Results in a 5 layer TCP/IP model
TCP/IP
Layered Architecture
7
Layered Architecture
Each layer operates independently of each other layer
Each layer below provides a protocol specification which provides a service to the layer above
The road provides a service for the tires of your car
Protocol – how to build a road to support weight of users
The tires provide a service for the car
Protocol – size and thickness of the rubber
The car provides a service for the driver
Protocol – how to drive the car and rules of the road
Road
For example, when driving your car from point A to point B, the road provides a service for your tires. The protocol specification would define how a road should be built to support the number of users. The tires provide a service for the car by supporting the weight of the care and moving it to the destination. The protocol specification would define the circumference of the tires and the density and thickness of the rubber to support the weight of the car. Lastly, the car provides a service for the driver. The car protocol would specify the frame format of the car such as size, shape and colour, and how to convert actions of the driver, such as steping on the brake pedal to slow down, or turning the steering wheel. Lastly, the driver would include protocol specifications on using the car, high performance driving, city-driving, leisurely driving and the rules of the road. The driver, for example, wants to go in a different direction, so he/she turns the steering wheel. The steering wheel is connected to the car’s tie-rod which rotates and turns the front axle. The change in direction has no effect on the job of the tires. They continue to roll and move the car regardless of the direction or road condition. This layered and independent architecture makes programming easier and modular. If you wanted to build a city-driving user-map, you don’t need to know how to build a road or how to build a car. The lower layers, acting independently, provide all of the essential services for the driver to reach his/her destination. One last point, the driver must determine “how” to get to point B from point A. For example, if the driver wanted to take a leisurely drive, he/she would hop into the care, begin driving, and not really caring if he/she arrives at the destination. This is analogous to sending an email with UDP (User Datagram Protocol) which is not connection oriented and does not guarantee delivery-unreliable connection. When a driver wants to guarantee arrival at their destination, he/she will use a map or GPS which provides an established route to the destination. This is analogous to using TCP (Transport Control Protocol) to retrieve a web page which is connection-oriented establishing a route from client to server, and guaranteeing delivery – reliable connection. We will speak more about this later.
8
Layers & Protocol – Data Communication
Protocol – Collection of Rules for physical establishment of calls, identification, transmission of data, acknowledgement & termination
Protocol -universal standardization of information exchange over the PC from different vendors, platforms and network configuration
Protocol – designed in forms of layer.
Layer- a specific piece of software following some pre-defined rules of functioning.
Unique functions or process are designated to different layers in a communication protocol and layers in conjunction provide the efficient communication.
Layers are implemented at the source and destination and any piece of hardware which provides interfacing between the two
Source
Destination
Message
Protocol
Media
Layers & Protocol – Data Communication
The components involved in data communication are: Source, Destination, Media, Protocol and the message exchanged between the source and the destination.
Protocols are the set of rules or the procedure which is followed by communicating devices while exchanging the data
The rules that allow two entities (source, destination) to exchange some message (text, audio, video) using a physical quantity (medium-wired/wireless) is known as protocol. Protocols provide the common standardization and flexible interconnection of communication among different platforms and different kinds of computing equipment (interoperability) around the globe and locally. It is needed for establishing a connection, identification of the source and destination, transmission of data, acknowledgement of reception of data and termination of connection between the entities involved in data communication operation.
Layer is a specific piece of software following some pre-defined rules of functioning in data communications. Unique functions or process are designated to different layers in a communication protocol and layers in conjunction provide the efficient communication. Layers are implemented at the source and destination and any piece of hardware which provides interfacing between the two. It is the task of the layer is to ensure that data is physically delivered to the destination without any errors.
9
TCP/IP 5 Layer Model
Application Layer – controls user interface
creates the message, defines the protocol, converts message to digital format. It controls user interface
Transport Layer -controls data transportability from system to system
adds transport header sequence number\source destination ports fragmentation/reassembly, flow control.
Internet Layer – control efficient routing and paths in the network
adds IP header, source\destination IP addresses, routing.
Data Link Layer – control error free transmission over physical layer
adds Data Link header, source\destination MAC address and trailer, FCS for error checking
Physical provides – interconnection through physical media
encoding\decoding and transmits bits across physical medium
Notice layers are numbered from the bottom up.
Notice the type of data at each layer
Application – Message
Transport – Segment
Internet – Datagram/Packet
Data Link – Frame
Physical – Bits
10
Application Layer
All Internet applications connect using SOCKETS, and IP address and a port number
e.g. 64.124.38.2:80
The Hypertext Transfer Protocol (HTTP) transfers files on the World Wide Web – port 80
The File Transfer Protocol (FTP) transfers individual files to and from servers –ports 20 and 21
The Simple Mail Transfer Protocol (SMTP) transfers mail messages and attachments- port 25
Additionally, the following Application layer protocols help manage TCP/IP networks:
The Domain Name System (DNS) protocol resolves a host name, such as senecacollege.ca, to an IP address and copies name information between DNS servers –port 53
The Routing Information Protocol (RIP) is a protocol that routers use to exchange routing information on an IP network –port 520
The Simple Network Management Protocol (SNMP) collects and exchanges network management information for centralized administration of network devices such as routers, bridges, and servers – port 161
11
Message Transmission Using Layers
12
Application Layer
Protocol Specification
13
Protocol Specification
Message Sequence – Who initiates the communication?
Message Type – What task or command is to be performed?
Message Syntax – What is the meaning of the message?
Type of Connection – What type of connection is needed by the protocol, reliable or unreliable?
Message Sequence – HTTP
The client sends an HTTP request message to the server
The server returns an HTTP-response message which answers the client request, or sends an error message
The server can not initiate a conversation with the client
1 Client sends HTTP-Request
2 Server sends back an HTTP-Response
Message Type- HTTP
GET – commands the server to return a file at the URL – “Get the File”
POST – uploads a file, usually containing online form data, to the webserver for processing – “Here is the file”
There are also other file types not discussed here
1 Client sends HTTP-Request
2 Server sends back an HTTP-Response
When humans talk, it is usually for a purpose: What time is it? Where are the washrooms? What do you take in your coffee? There is no limitation on the questions we can ask and the order in which we ask them. Humans are intelligent, software protocols are not.
16
Message Syntax – HTTP
Refers to the organization of the message
Header – something added before the data field
Data – is the purpose of the message created by the application layer
Trailer – something added after the data field, usually a FCS for error checking
Not all packets will have a data field and most do not have a trailer field.
Data
Trailer
Header
Message syntax refers to the organization of the message. In conversation, human’s tends to be informal, even chaotic. Software must use very rigid and exact syntax. In general all messages have three parts: a header, data and trailer fields. The data field is the purpose of the message, created by the application layer; it contains the content to be delivered. The header is simply everything that comes before the data field and the trailer field is everything that comes after. Not all messages, however, will have a trailer field. HTTP messages
17
HTTP Request Message
GET method which requests a path to a file. Notice this line and all other lines end with a character return and a new line [CRLF]
host to which the request is sent people.senecac.on.ca
host browser and operating system used to send the request
language to be used in this case English US
type of compression to use, such as gzip or deflate
The sixth line specifies the type of TCP connection
in this case “keep-alive” means to keep the connection active over multiple request-response cycles. Without this, the TCP connection would end after each request-response cycle and would have to be re-established. Notice, the 2 blank lines at the end of the request; these lines are required and indicate the end of the header field. The data field is empty.
18
HTTP Response Message
The first line begins with HTTP/1.1 which indicates the server has a compatible version. The 200 is a success code that the desired file is returned. The browser actually ignores this code; it is designed for humans to indicate the request was successful.
The next line indicates the server that returned the request
The next line gives the date and time the request was returned.
The next heading specifies the content type which was returned, in this case an image file.
Again, the end of the header is marked with 2 blank lines followed by the content returned in the data field (not shown). There is no trailer.
Please note that each student will participate in a group project exploring any Internet protocol, EXCEPT HTTP. The group will present a 4-6 page report following the same outline as discussed here with HTTP and will post the project on the web using Blackboard’s portfolio tool.
19
Message Encoding
UTF-8
20
UTF-8 Encoding
Application layer main job is to convert message into a digital format.
Application data may be text, image, or video.
A conversion table is used to convert a byte into a unique numerical value; a process called “encoding”. Each encoded value is then converted to a series of 0s and 1s.
Unicode (Universal Encoding) can represent all of the written languages in the world and is the de facto standard for text on the web with over 86.2% of all web pages in 2016 use UTF-8
UTF-8 Encoding
UTF-8 (Universal Coded Character Set + Transformation Format – 8-bit.)
a variable-length encoding system using 1 to 4 “octets” (1 byte is called an octet)
The first octet representing the first 128 values is identical to ASCII, making UTF-8 a superset of ASCII and safe to use with programming languages that interpret only one byte encoded characters.
A Unicode character is represented by writing “U+” followed by a hexadecimal number
When one byte is used for ASCII values the high order bit begins with a “0”. Characters above 127 in value will use more than one byte
The number of “1s” in the first byte indicates the number of octets used for encoding, followed by a “0”. Each subsequent byte has a “continuation marker” of “10” at the beginning of the byte. Refer to the table below.
Some people mistakenly believe that Unicode is simply a 16-bit code with 65,536 possible characters. This is completely incorrect. UTF-8 has a character space, called code points, of 1,112,064 valid characters.
22
UTF-8 Encoding
UTF-8 is “self-synchronizing”. The high order bits of every byte determine the type and number of bytes for each character. Since there is no overlap of values, a receiving computer can reevaluate a transmission by backing up at most three bytes to determine the start of a character.
Bits Decimal
Range First Code Point Last
Code
Point Bytes used Byte1 Byte2 Byte3 Byte 4
7 0-127 U+0000 U+007F 1 0xxxxxxx
11 128-2,047 U+0080 U+07FF 2 110xxxxx 10xxxxxx
16 2,048-65,535 U+0800 U+FFFF 3 1110xxxx 10xxxxxx 10xxxxxx
21 65,536-1,112,064 U+10000 U+1FFFFF 4 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
Some people mistakenly believe that Unicode is simply a 16-bit code with 65,536 possible characters. This is completely incorrect. UTF-8 has a character space, called code points, of 1,112,064 valid characters.
The UTF-8 is standard actually uses a maximum of 6 bytes to encode characters, but the W3C has restricted the standard to 4 bytes at the present time
23
UTF-8 Encoding: Examples
To encode UTF-8 you need to understand decimal to digital and hexadecimal conversion
.Example 1: we will encode the, ASCII character “a”, decimal value of 97
Example 2: we will encode the Unicode value U+20AC, representing the Euro symbol, €.
Convert the number to binary by dividing by 2, till 0 or 1 in the remainder
Write the remainders in reverse order
Convert the binary value to hexadecimal
Steps:
24
Example 1: Converting Decimal Value to Hexadecimal
Convert the number to binary by dividing by 2
Write the remainders in reverse order
Example 1: Converting Decimal Value to Unicode
Convert the binary value to hexadecimal by grouping the binary value into 4 bits and determining each 4 bit value
Write the Unicode code point for the hexadecimal value – 97 decimal = U+0061
Example 2: Converting Unicode Value U+20AC to UTF-8 Encoding
Unicode characters are represented by hexadecimal notation
The value U+20AC is only the bit sequence or the code point to represent the Euro symbol “€”
It is not the hexadecimal UTF-8 encoded string created by the application layer.
Convert the code point to binary
Encode the binary value into a UTF-8 multi-byte sequence and
Then convert the sequence back to hexadecimal.
Example 2: Converting Unicode Value U+20AC to UTF-8
To convert the code point to UTF-8 encoding, we must first :
Convert the code point to binary
Encode the binary value into a UTF-8 multi-byte sequence and
Then convert the sequence back to hexadecimal.
Example 2: Converting Unicode Value U+20AC to UTF-8
If you need to convert hexadecimal to decimal follow the steps below:
Example 2: Converting Unicode Value U+20AC to UTF-8
Example 2: Converting Unicode Value U+20AC to UTF-8
The Unicode character U+20AC has a hexadecimal UTF-8 encoding of 0xE2 82 AC
Encoding Alternatives
Bits Alternatives Examples
1 2 Yes\No Male\Female
2 4 Married, Single, Divorced, Widowed
3 8 Number of Company Departments
4 16 Top 10 Google Searches
8 256 Number of Users in company
16 65,536 Number of colour levels
32 4,294,967,296 Number of IPv4 Addresses
Sometimes the application layer will use a number of bits to represent alternatives, such as a field representing the type of message, or the protocol to use
Many applications today involve voice and video. To encode an analog signal a special electrical circuit is used called a CODEC (Encoding\Decoding). This circuit measures the amplitude of the wave and converts it into a digital value, called encoding. When the digital signal is converted back to an analog signal, to play through a speaker, the signal is decoded.
Suppose, you needed to represent 4 alternatives; the bit values of 00, 01, 10 and 11 could be used
The rule is a = 2b; “a” is the number of alternatives and “b” is the number of bits needed. How many bits are needed for 12 alternatives? You would need 24 which give 16 alternatives, of which 4 will be unused. You should memorize the number of alternatives that can be represented by 4, 8 and 16 bits, since these are common field sizes. Each added bit doubles the number of possibilities. Contrarily, each bit subtracted cuts the number of alternatives in half.
Examples are suggestive only – add your own.
32
TCP/IP Protocol Stack in Action
After the application layer has converted the message into a digital format, and followed the protocol specifications the message is passed to the Transport layer to begin its outgoing transmission. The message is placed in the Transport’s data field. The process of placing a message in data field of another layer is called “encapsulation”. It is similar to placing an envelope inside anther envelope. The Transport layer adds a header to the TCP packet and passes the packet to the Internet layer. The Internet layer process encapsulates the packet inside an IP datagram by adding an IP header. The datagram is then passed to the Data-Link layer which encapsulates the datagram inside an Ethernet frame (or any other LAN or WAN technology). The frame is passed to the physical layer which converts the bits of the frame into signals for outgoing transmission. The physical layer does not do encapsulation. It connects the sending computer to a switch, router or host. While we think of a direct connection from application layer to application layer on the receiving computer, the connection is actually virtual. The only real connection is at the physical layer. The message travels down 5 layers (from the application layer), across the physical layer and up 5 layers to the application layer on the receiving computer.
At the receiving computer, the message is received by the physical layer, decoded and passed to the data link layer. The frame is checked for errors and if no errors the FCS, trailer is removed. The header is checked to make sure that the MAC address matches the destination computer, and if correct, the frame header is removed and passed to the Internet layer. The Internet layer checks to make sure that the destination IP address is correct and then removes the header and passes the datagram up to the transport layer. The transport layer reassembles the packets if necessary and arranges the packets in the correct order and passes the packets to the application layer which displays the original message.
33
Standard Organizations
International, National, Industry, Professional and Internet
34
Standard Organizations
ITU-T is an agency of the United Nations
ISO is an independent organization made up of all the national standard organizations
ANSI and Standard Council of Canada (SCC) – independent national bodies often work together
EIA\TIA are separate industry organizations that often form joint standards. IEEE is the largest professional organization comprised of engineers
IETF is an independent body responsible for the Internet
Standards are made by fact called “de facto” or by law called “de jure”. If the majority of the market spontaneously begins using a standard, this is an example of a de facto standard. For example, no standard body legislated that UTF-8 encoding had to be used. People just started using it because of its benefits and now it dominates the web.. Most standards, however, are legislated by a standard making body.
It is important for programmers to have a brief understanding of standard making bodies, because if you are working on a new project it is common practice to design a project to conform to some standard technology. You will need to know where to find the appropriate standard for your project.
ITU-T – International Telecommunications Union – Telecommunications division
The International Organization for Standardization (ISO, notice the name does not match the acronym) is an independent non-governmental international body make up of members of other national standard bodies. The ISO has over 162 participating countries which review the standards of national bodies and adopt those that promote international trade. They make standards on all topics, from food and safety management , factory quality control, to Information Security.
National standard making bodies such as American National Standards Institute (ANSI) in the United States and the Standard Council of Canada (SCC) in Canada have three principal roles. Their primary role is to promote national businesses. When technology, developed by local business, becomes a national or international standard, the Gross National Product increases because other companies pay a royalty to use the technology. As more people buy the technology the local business expands and employees more people. The second role is to protect national health and safety. Private businesses are not motivated to spend money to develop national standards for health and safety; this is an important governmental role. Lastly, national standard making bodies coordinate the development of standards by other industry or professional groups. They are always on the lookout for promising technologies.
ANSI is a private non-profit organization, but its membership consists of private and public organizations. ANSI controls the standards for fiber-optic networks and other MAN technologies. The SCC is a crown corporation and also operates independent of government, but reports to Parliament through the Minister for Industry. The SCC has always worked closely with ANSI and in 2015 signed a Memorandum of Understanding (MOU). As John Walter, the CEO of SCC stated:
SCC and ANSI are working together to break down barriers between Canada and the US. This MOU is a pivotal step toward making progress on joint Canada-US standards, testing and certification procedures. The reduction of duplicative standards, testing and certification requirements between our two countries will enhance our competitiveness.
Industrial standards are developed by the manufacturers and retailers of electronic and telecommunications equipment. The Electric Industry Alliance (EIA) and the Telecommunications Industry Association (TIA) are the principal players. Their goal is to develop public awareness, education and standards for the physical interface and electronic signaling. Often the two associations will produce joint standards EIA/TIA on interfaces, such as EIA/TIA 586A cable interface.
John Walter, CEO, SCC from the www.scc.ca web site.
The Institute of Electrical and Electronic Engineers (IEEE) is the largest professional standard making body in the world. With branch members in all industrial countries IEEE standards have international compliance. The IEEE is responsible for all LAN technologies such as 802.3, Ethernet and 802.11 wireless LAN (WLAN) and 802.15 PAN (Personal Area Network) which connects Bluetooth devices to a wired network. Since the bottom two layers are OSI layers, technically LAN standards are not official until adopted by the ISO.
Lastly, the Internet is a non-governmental group with membership open to any user of the Internet. The principal agency is the IETF, Internet Engineering Task Force. This group is responsible for TCP/IP and issues standards called RTF, Request for Comment.
35
Summary
In the late 70’s the ISO developed a 7 layered architecture called Open System Inter-Connection; this model has not replaced TCP/IP, but has modified it by adding the physical and data links to create a 5 layer TCP/IP protocol stack.
The TCP/IP protocol stack is a layered model with each layer working independently of the other layers. Each lower layer also provides a service to the layer above. This modular approach allows changes in cabling with affecting the higher layers.
Each layer also has a protocol specification. The application layer protocol specification affects the sequence, type and syntax of the messages used in data communications.
The most important job of the application layer is to convert the message into a digital format. The most common format today is UTF-8 which is 100% compatible with ASCII 8 bit character set and is self synchronizing.
Lastly, standards are created by standard organizations which exist at the national, international, industrial, professional and Internet areas. Standards must set in advance of data communication and networking, which requires standard organizations to work together
NRZI – Non Return Zero Inverted
Baud rate is how many times the signal changes per second
Bit Rate is how many bits are sent per second