COMP3310/6331 – #20
Measuring, monitoring and SNMP
Dr Markus Buchhorn: markus.buchhorn@anu.edu.au
Network monitoring
• Measuring networks – and monitoring – What do you measure
– How do you measure it
• Want to know:
– How busy is some/all of the network?
– Is there congestion (somewhere)?
– Are there errors?
– Is the hardware/software ok?
– Is there a bug in the network? (literally?!?)
– Has something changed, for the worse, or the better? – Are applications being fed the right packets?
– Is routing behaving as expected?
–…
2
Performance over time
• Capacity planning • Outages
• Patterns
3
Performance at a moment: Network status
And:
What’s out there? Network Discovery
4
Network feedback
• ECN – Explicit Congestion Notification
• ICMP – Internet Control Management Protocols
– Used passively and actively (ping, traceroute, …) • TCP ACKnowledgements
• Application measures
•…
• No unified view
• No aggregated view, in space or time
5
Two domains
• Within your administrative domain (interior)
– You have authority
– Get information from everywhere on your network – Put some software on each device
– Probe, measure, scan, …
• Beyond your administrative domain (exterior) – No authority
• Except maybe a contract?
– Ask somebody else to put some software on each device, and share
6
Simple Network Management Protocol (SNMP)
• Design requirements: We want…
• Reach everywhere
– All sizes, types of devices
• Switches, routers, access points, printers, servers, …
• Support devices that are too small, too simple, too hard, too old, …
• Lightweight
– no interference on device
• Operate when things are under stress
– Identify what is struggling/failing, and when
– Help to fix/improve things
• Scale to large number of devices and parameters
– Global naming, delegated, vendor-independent, extensible
• Provide both queries/response and command/control
• And add some trivial security and upgrade it much, much later
7
SNMP
• An application framework
• For managing/monitoring network resources
• Components of SNMP:
– SNMP agents
– SNMP managers
– Management information bases (MIBs) – SNMP protocol itself
8
SNMP components
• Agent: software on the equipment
– maintains configuration and current state in a database. – Proxies: an agent that talks with non-SNMP devices
• Management Information Bases (MIBs) describes the database. – MIB, MIB-II (RFC 1213) – and millions more
– Structure of Mgmt Info (SMI) defines sets of related objects in a MIB
• Manager: application that contacts an agent
– to query or modify the database at the agent. – Part of Network Management Systems (NMS)
• SNMP protocol:
– SNMPv1, v2(*), v3
9
Information design for lightweight SNMP agents
• No rates, no calculations
• No absolute clocks
• No history
• Just
– Countersandgauges, – Timesincestart-up
– Strings,Identifiers
• “Timeticks”,in1/100thssec.
• Command/controlthroughvariablesetting
10
SNMP protocol
SNMP protocol
MANAGER AGENT
12
SNMP Proxies
SNMP messages
• SNMP/UDP is connectionless
– Use a request ID to maintain a session
• SNMP messages are ‘protocol data units’ (PDUs)
– Different versions of SNMP use the same PDU for different messages
• We’re still living through that pain…
• Messages have particular capabilities (SNMPv1): – Get – the value of a object from an agent
– Set – the value of a object from an agent
– Notify – a manager that the agent has had an event
14
SNMP(v1) Protocol
•
– – – –
•
–
On-demand:
Get-request: Request the values of one/several objects Get-next-request. Requests the value of the “next” object. Set-request. Modify the value of one or more objects Get-response. Agent response to a request.
Triggered:
Trap: A notification from an agent to a manager, some event at the agent.
15
Traps
• Traps are sent asynchronously by an agent to a manager
• 6 core traps: – linkDown: – linkUp:
– coldStart:
An interface went down
An interface came up Unexpected restart (system crash)
– warmStart: Expected restart (manual reboot)
– AuthenticationFailure: Somebody tried to query, but …
– egpNeighbourLoss: Link is up but my neighbour has gone
• And ~232 others (vendor specific)
16
Format of SNMP (v1/v2) Packets
• Get/Set:
Identifies how you “belong”
Version
Community
SNMP PDU
PDU Type
Request ID
Error Status
Error Index
Object 1, Value 1
Object 2, Value 2
…
Unique ID to match requests with replies
What went wrong, and where
Sequence of name-value pairs
17
SNMP community
• SNMPv1defines“communities”
– specifyaccesstospecificvariablesets – read-write,readonly,none
• EachSNMPmessageincludescommunityname – Likeapassword
– Unencrypted!!
• Typicalvalues:
– Read-only:“Public” – Read-write:“Private”
• Slightenhancement:agent/managerrelationship – IPaddressofpermittedmanagers,storedonagent
• Firstthingfixedinv2…
18
SNMP Versions
• Three versions in use today:
– SNMPv1 (1990)
– SNMPv2c – [and three more] (1996)
• Adds “GetBulk” function
• Adds federated monitoring capabilities (manager to manager) • Adds TCP transport option
• Adds 64bit counters
– SNMPv3 (2002)
• SNMPv3 started from SNMPv1 (and not SNMPv2c) • Addresses security
• All versions are still used today.
• Many SNMP agents and managers support all three versions.
19
SNMP Security
• SNMPv1 uses “community” strings for authentication – In plain text without encryption
• SNMPv2 was supposed to fix security problems, but effort derailed – The “c” in SNMPv2c stands for “community”??
• SNMPv3 has key security features:
– Ensure that a packet has not been tampered with – Ensures that a message is from a valid source
– Ensures that a message cannot be read
(integrity), (authentication) (privacy).
20
SNMPv3
• Has three security levels:
– Depending on how you connect – you get more access rights
• noAuthNoPriv: Authentication by matching a user name.
• authNoPriv: Authentication with message digests.
• authPriv: Authentication with message digests, and encryption 21
SNMPv3 Manager
COMMAND GENERATOR
N O TI F IC A TI O N RECEIVER
MESSAGE PROCESSING SUBSYSTEM
SNMPv1
SNMPv2C
SNMPv3
OTHER
PDU DISPATCHER
MESSAGE DISPATCHER
TRANSPORT MAPPINGS
SECURITY SUBSYSTEM
COMMUNITY BASED SECURITY MODEL
USER BASED SECURITY MODEL
OTHER SECURITY MODEL
SNMPv3 Agent
MANAGEMENT INFORMATION BASE
COMMAND RESPONDER
NOTIFICATION ORIGINATOR
ACCESS CONTROL SUBSYSTEM
VIEW BASED ACCESS CONTROL
MESSAGE PROCESSING SUBSYSTEM
SNMPv1 SNMPv2C SNMPv3 OTHER
PDU DISPATCHER
MESSAGE DISPATCHER
TRANSPORT MAPPINGS
SECURITY SUBSYSTEM
COMMUNITY BASED SECURITY MODEL
USER BASED SECURITY MODEL
OTHER SECURITY MODEL
What are we GET/SETting in those packets?
• Values stored in a Management Information Base (MIB)
– Collected under a Structure for Management Information (SMI)
• Written in a formal language (ASN.1) – A formalism, rather than a language
• Field day for informaticians, logicians and other purists…
24
25
On Counters and Gauges…
• Reading Counters/Gauges tell you about “now”
– Counter e.g. packets on an interface (can wrap)
– Gauge e.g. memory/disk space (ranges between zero and
• Agents don’t have history, and don’t calculate rates/changes – Agents only have a temporary clock – Time since boot
• Managers have to ask more than once, and make assumptions
– Counter doesn’t change = World hasn’t changed
– Gauge doesn’t change = World may have changed, or not, between requests
– MIB designers might need multiple fields/types for related information
26
ASN.1 Know it exists and where to look it up…
• Abstract Syntax Notation One (1980’s) – predates XML, etc. • Formal description of data structures, message formats
– Type, length, value (TLV)
• Predefined basic types
– BOOLEAN, INTEGER, OCTET STRING, BIT STRING, REAL, – ENUMERATED, CHARACTER STRING, OBJECT IDENTIFIER
• Constructed types
– SEQUENCE, SEQUENCE OF, CHOICE
– Arbitrary nesting of types and sub-types
• Encoding types (10+) = TLV to bytes – we’ll stick with ‘BASIC’
27
ASN.1 OBJECT IDENTIFIER (MIB)
.
iso(1)
org (3) dod (6)
• Define an information object and reference
• Managed at the international level
• internet OBJECT IDENTIFIER ::= { iso org(3) dod(6) 1 } • Globallyunique
root
28
OID Organisation
• Tree hierarchy – like DNS
• Each OID is a node in the tree.
• Most internet stuff is 1.3.6.1.2.1.xyz
• Manufacturers can add product specific objects to the
• SNMP uses OID for reference
• MIBs map OID to readable form – And specify their type, etc.
.
iso(1)
org (3)
dod (6) internet (1)
mib-2 (1)
at (3)
ip (4)
ipForwDatagrams (6)
root
mgmt (2)
experimental (3)
directory (1)
system (1)
private (4)
icmp (5)
udp (7)
snmp (11)
interface (2)
tcp (6)
egp (8)
transmission (10)
29
ASN.1 examples
• Type definitions
– NumberofStudents ::= INTEGER
– PassOrFail ::= BOOLEAN
– GradeType ::= ENUMERATED {A, B, C, D, E, F} – PointsScored ::= REAL
– Image ::= BIT STRING
– Data ::= OCTET STRING
• Value definitions and assignments
– studentsFridaySession NumberofStudents ::= 9 – passCourse PassOrFail ::= TRUE
• Combine type/value definitions
– StudentType ::= INTEGER { ugrad (0)
}
ms (1) phd (2)
30
ASN.1 string examples
• Access ::= “read-only” | “read-write” | “write-only”
| “not-accessible“ • Status ::= “mandatory“
| “current”
| “optional”
| “obsolete”
31
A MIB “object”
— The Interfaces group
— Implementation of the Interfaces group is mandatory for all systems.
ifNumber OBJECT-TYPE
SYNTAX INTEGER
ACCESS read-only
STATUS mandatory
DESCRIPTION
“The number of network interfaces (regardless of
their current state) present on this system.”
::= { interfaces 1 }
Variable names are aliases for digit strings (defined by MIB) interfaces defined in MIB as 1.3.6.1.2.1.2, so ifNumber = 1.3.6.1.2.1.2.1
.
iso(1)
org (3)
dod (6) internet (1)
mib-2 (1)
at (3)
ip (4)
ipForwDatagrams (6)
root
mgmt (2)
experimental (3)
directory (1)
icmp (5)
private (4)
system (1)
udp (7)
snmp (11)
interface (2)
tcp (6)
egp (8)
transmission (10)
ifNumber (1)
33
MIB-2 object counting packets
ipForwDatagrams OBJECT-TYPE SYNTAX Counter
ACCESS read-only
STATUS current
DESCRIPTION
“The number of input datagrams for which this entity was not their final IP destination, as a result of which an attempt was made to find a route to forward them to that final destination. In entities which do not act as IP Gateways, this counter will include only those packets which were Source-Routed via this entity, and the Source- Route option processing was successful.”
::= { ip 6 }
Aka 1.3.6.1.2.1.4.6
34
More on the interfaces (MIB-II)
Name
Description
ifMTU
Maximum packet size
ifSpeed
Bits/sec
ifPhysAddress
e.g. MAC address
ifOperStatus
Up(1), Down(2), Testing(3)
ifInErrors
# incoming packets discarded due to errors
ifInDiscards
# incoming packets discarded due to buffer overflow
ifOutQLen
# packets in outbound queue
ifInUcastpkts
# incoming packets received
35
Why?
• OIDs provide global uniqueness – and extensibility
• OIDs provide human-readable-names for tree-position-identifiers
• Also: ASN.1 does not offer tables – But humans need them
Interface #
IP address
State
Packets
Errors
Rate
1
150.203.1.1
Up
1172
5
100Mb/s
2
130.56.3.1
Up
1234
3
100Mb/s
3
197.197.4.1
Down
5678
4
100Mb/s
4
197.197.5.1
Up
8451
197
1000Mb/s
5
8.8.8.1
Up
9191
2
10Mb/s
36
Tables and GetNext
• Each table cell has a 1.3.6.1.2.x.y.z.abc.label identifier
• Rows in a table get sequential entries based on the index
– E.g. Interface number
• Manager doesn’t know how many rows (interfaces) there are
– There is no ‘row/column-count’. Don’t need it. May change anyway!
• • • •
• This works even if you don’t know the names/columns/rows – Lexicographical Order for OIDs 37
Get (“Interface.1.ipAddress”)interface.1.ipAddess = 150.203.1.1 Get-next (“Interface.1.ipAddress”)interface.2.ipAddress = 130.56.3.1 …
Get-next (“Interface.5.ipAddress”) something else in the MIB
Ok, but…
• Repeated Get-next:
– Lots of extra there-and-back traffic
– More state to maintain/evolve in Manager (row#/column#)
• SNMPv2 introduced Get-Bulk request
– Get-Bulk(“interface”) every row, every column
– But only one UDP packet comes back – Error response “tooBig” (64kB UDP limit)
38
Review: Performance over time
MRTG, Cacti, Nagios – as monitoring/graphing tool
39
Network status
Network Weathermap (PHP, reads MRTG data)
Vendor tool
(can draw layouts right)
40
SNMP beyond my domain?
• SNMP in the wide area is … unwise
– SNMPv1/v2 agents should not be visible. Ever.
– Lots of traffic
– Easy to scan/map network
• Becomes a human problem
– Need to ask for favours
• Beacons (e.g. multicast)
• perfSONAR
• Looking Glass
– Remote login
– Limited (read-only) queries
– Various ISPs
41