VULNERABILITY ASSESSMENT
7CCSMSEM Security Management Dr. Jose M. Such
Introduction
• Vulnerabilities are those things that can be exploited in order to breach CIA-N
• This may be in: • Technology
• Processes
• People
• Main focus in the industry is the wide reporting of the technical
• People/processes company specific(?)
• Vulnerabilities need to be known to be exploitable
Vulnerability Trends
Source: http://www.sans.org/top-cyber-security-risks/trends.php.
Technical Vulnerabilities
• Vulnerabilities listed in various places
• NIST National Vulnerability Database (https://nvd.nist.gov/) • Open Source Vulnerability Database
• US-CERT Vulnerability database
• UK-CERT / NCSC CiSP (https://www.ncsc.gov.uk/cisp)
• Most vulnerabilities specified as a Common Vulnerability and Exposures format (CVE)
• http://cve.mitre.org/
• Common Vulnerability Scoring System – https://www.first.org/cvss/
Hardware Vulnerabilities
• Covers phones, routers, switches etc.
• Can be a hardwired problem
• This includes firmware
• Often concerns about upgrading
• Vendors are often slower at responding than SW vendors
Application Vulnerabilities
• This is where hackers are going
• OS vendors getting better at writing secure code, security auditing,
patching, secure config out of the box
• Applications are the most vulnerable
• A system is as strong as it’s weakest link
• Web and Mobile applications are the most fertile ground
• Everything is a web or mobile app these days • Applications get poorly patched
System Configuration
• This is a difficult area as extensive knowledge is required to do a good job
• Most systems come preconfigured in a weak state • Default configuration have vulnerabilities
• Similar standard to CVE, Common Configuration Enumeration
• http://cce.mitre.org/
• https://nvd.nist.gov/config/cce/index
Social Engineering
• Vulnerabilities exist in people and processes
• Helpdesks are there to assist people
• People don’t really care about what happens to their corporate information
• http://www.theregister.co.uk/2003/04/18/office_workers_give_away _passwords/
• Social Engineering Toolkit • Automated attack system
How Do You Find Vulnerabilities?
• Vulnerability Assessment:
“The systematic examination of a system to identify those critical infrastructures or related components that may be at risk from an attack and the determination of appropriate procedures that can be implemented to reduce that risk.”
– American Nation Standard T1.523-2001
Assurance Techniques
• Techniques that allow us to establish the level of assurance we have about how secure a system is
• The output of these techniques contributes to Vulnerability Assessment
• Some techniques also contribute to other risk assessment steps like asset identification (architectural review) and threat assessment
Assurance Techniques
Review
Simulated Attack
Review of
D oc um en te d P ol ic ies , Procedures, and P ro ce sses
Review of
C li ent – C om pl et ed Self-Assessment Form
Te st
Optional Contributing Assurance Technique Optional Parallel Assurance Technique
Red Team Ex er ci se
Cryptographi c Val id ati on
Architectural Revie w
Threat Ass essm en t
Penetration Test
Formal Ver i fic ati on
C on fi gur at io n Review
Source Code Revie w
Vulnerability Scan
Em ana ti on Security Analysis
O b s e rv e
I n te rv i ew
Static Analysis
Social En gin eer i ng
Independent Validation
Dynamic Analysis
Fuzzing
Public Review
Witnessed Test
Oral
Pape r-Based
Pape r-Based
Em pl oym en t
Individual Competence
VirtualLab Ex ami n ati on
Ex ami n ati on s Ex ami n ati on Ex ami n ati on
History and
(Viva Voce) (Narrative) (Multiple Choice)
Qu ali fic at io n
Review
• Review of Documented Policies, Procedures, and Processes
• The process of analysing the documented specifications (e.g., procedures and security properties) and processes (e.g., managerial) for a component or system under assessment.
• Review of Client-Completed Self-Assessment Form
• An analysis of a client submitted review of their implementation of assurance targets as set out within an assurance scheme. Self- assessment forms typically consist of a multitude of questions that a client must answer is multiple choice or narrative form.
• Threat Assessment
• Amulti-stageprocessusedtoidentifyandrankthethreatstocomputer software, a component, or IT system. It comprises strategies or pathways used to determine the credibility and seriousness of a potential threat, as well as the likelihood that it will be carried out in the future
Review
• Architectural Review
• An analysis of the components (type, quantity, etc.) and their relationships within a piece of software, component, or system to determine if their implementation meets a desired security policy.
• Configuration Review
• A review of the way a system or its software has been configured to see if this leads to known vulnerabilities. Configuration reviews can be passive (e.g., manually checking software versions for known vulnerabilities) or active (e.g., automated build review scanners).
• Source Code Review
• The examination of source code to discover faults that were introduced during the software development process. Source code reviews are predominantly manual; however, they may be supplemented with automated techniques (e.g., using static analysis tools).
• Observe
• The process of watching a live, operational system to identify real-
world deviations from documented assurance targets.
• Interview
• The process of questioning one or more individuals about security- related matters within the organisation being assessed through any medium (e.g., in person or virtually).
Test
• Penetration Test
• Asimulatedattackonacomponentorsystemusingsimilartechniquesto that of a real- world malicious attacker. A penetration test may build upon a vulnerability scan; however, it differs in having an implicit or explicit goal that the assessment attempts to realise (e.g., compromise sensitive data or obtain a certain level of network access). Typically this requires vulnerabilities to be exploited, which would not be undertaken within a vulnerability scan.
• Red Team Exercise
• Asimulatedattackonasystemthatisgivenmorefreedomthanisavailable during a penetration test, in order to more realistically simulate a real-world malicious attacker. This freedom is given in terms of the engagement’s duration (e.g., often months in duration), available human resources (e.g., large teams built around individuals with different specialisms), allowed use of tools (e.g., a heavy use of social engineering is common), and restriction of defender knowledge to test their day-to-day responses to cyber threats.
• Vulnerability Scan
• The process of using an automated scanner on a web application or network to identify vulnerabilities. Discovered vulnerabilities are not exploited.
Test
• Social Engineering Testing
• An attempt to manipulate one or more human users into performing an action that does not conform to operational procedures. This can be conducted in a manner that is goal-based (e.g., access data) or audit- based (e.g., the percentage of a department vulnerable to a spear phishing attack).
• Static Analysis
• Without executing computer software, static analysis attempts to debug and identify potential software vulnerabilities through an analysis of its source code. Static analyses are predominantly automated; however, they may contain some elements of manual interaction (e.g., in order to understand the context and implications of the results). Human-led analyses fall under source code review.
• Dynamic Analysis
• Once computer software has been executed, this technique attempts to debug and identify potential software vulnerabilities through active methods (e.g., inputting unexpected data through fuzzing) and passive methods (e.g., memory analysis).
• Fuzzing
• The process of injecting erroneous and unexpected data into an input field in order to trigger faults (e.g., crashes and exceptions) that could be leveraged to discover
software vulnerabilities. Fuzzing may be dumb (i.e., random) or intelligent (i.e., with a knowledge of the protocol being tested).
Test
• Formal Verification
• The use of mathematical techniques for assessing functional properties
of information and communication systems.
• Cryptographic Validation
• A method used to analyse a cryptographic algorithm and/or its
implementation within a component or system (e.g., entropy testing).
• Emanation Security Analysis
• One or more methods used to assess device emanations (e.g., electromagnetic or sound emanations) for the unintentional leakage and disclosure of information.
Independent Validation
• Third party involved • Witnessed Test
• The use of an independent witness to provide a second level of verification that the results of an assurance technique are as described.
• Public Review
• The process of opening a technology, component, or system to wider review by the public. Public reviews may be of documents (e.g., drafts of future cryptographic algorithms) or live systems (e.g., bug bounties).
Assurance Techniques
Review
Simulated Attack
Review of
D oc um en te d P ol ic ies , Procedures, and P ro ce sses
Review of
C li ent – C om pl et ed Self-Assessment Form
Te st
Optional Contributing Assurance Technique Optional Parallel Assurance Technique
Red Team Ex er ci se
Cryptographi c Val id ati on
Architectural Revie w
Threat Ass essm en t
Penetration Test
Formal Ver i fic ati on
C on fi gur at io n Review
Source Code Revie w
Vulnerability Scan
Em ana ti on Security Analysis
O b s e rv e
I n te rv i ew
Static Analysis
Social En gin eer i ng
Independent Validation
Dynamic Analysis
Fuzzing
Public Review
Witnessed Test
Oral
Pape r-Based
Pape r-Based
Em pl oym en t
Individual Competence
VirtualLab Ex ami n ati on
Ex ami n ati on s Ex ami n ati on Ex ami n ati on
History and
(Viva Voce) (Narrative) (Multiple Choice)
Qu ali fic at io n
• HOW DO WE CHOOSE WHICH ASSURANCE TECHNIQUES TO APPLY??
• Risk Management Decisions are cost-effective
• Should not the choice of Assurance Techniques be cost-
effective too??
• Economics of Assurance Activities project funded by UK Government – article on the outputs in KEATS
ques n
m
o
[ k
c
h p
v d
a
n n m
o n
ant
the ST) tion
ent ions und
cus ting 19], ills,
e is the as aid this For ing [6] and nce and t is on- ical ent l or mic
Figure 1: Methodology
• Survey with over 150 cyber security professionals
• “For each assurance technique, assume a commercial target of medium size. Examples: company with 250 employees; infrastructure with 16 external IPs or 150 internal IPs; web application with one database and 100 static or dynamic pages; product like a Firewall, Router or Switch.”
Economics of Assurance Project
Scheme
Scope
Target
CBEST/STAR CEH
CESG CAPS CESG CAS CESG CCP CESG CHECK CESG CLAS CESG CPA CESG CTAS CISSP
Common Criteria CREST
Cyber Essentials Cyber Scheme ISO/IEC 27001 PCI DSS
Tiger Scheme
National (UK) International National (UK) National (UK) National (UK) National (UK) National (UK) National (UK) National (UK) International International National (UK) National (UK) National (UK) International International National (UK)
Organisational security Individual qualification Organisational security Organisational security Individual qualification Individual qualification Individual qualification Organisational security Organisational security Individual qualification Organisation security Individual qualification Organisational security Individual qualification Organisational security Organisational security Individual qualification
Table 1: Assurance Schemes Reviewed
All of the gathered information was used to: (i) define a consistent and coherent assurance terminology to clearly sive define assurance schemes, targets, techniques, evidence and
Stakeholder Composition
3% (5)
3% (5)
9% (17)
18% (33)
67% (124)
Chief Information Security Officer
Competence Assessor (e.g., for qualifications)
Auditor
Information Security Manager
Security Practitioner (e.g., a penetration tester, security architect)
20+ yrs 15-19 yrs 10-14 yrs
5-9 yrs <5 yrs
0% 10%
20% 30%
17% 17%
19%
26% 21%
Characteristics
• Number of people required to conduct an AT
• Expertise required (practitioner, practitioner with
supervision, senior)
• Time required
• Effectiveness
• Cost
• Complementary ATs
Number of people and Expertise
computers & security 60 (2016) 117–133 125
Table 3 – Expertise required.
red.
Assurance technique Expertise required (%) Total
Expertise required (%) Total
P P(W) S Pr Resp.
P P(W) S Pr Review of [...] PPP
[...3]3Self As3s5essmen32t Form—
Ar4c8hitectu2ra9l Revie2w3 1
Con8figurat1io2n Revi7e1w 8
So2u3rce Cod4e4 Review33 —
Ob1s4ervatio2n5 46 14
Int3e0rview 41 25 4
Re1d3Team 1E9xercise5s8 10 Pen9etration14Tests 55 23
Vu1ln4erabil3it6y Scan45 5
So5c0ial Engi3n6eering14 —
Th2r0eat Ass3e2ssmen4t5 3
Stat5ic Anal2y2sis 59 14
Dy2n7amic A3n2alysis41 —
Fu1z7zing 36 48 — Fo2rm9alVer3i4fication34 2
Cry13ptograp2h5ic Vali4d8ation15 Ema7nation12Securit5y9Anal2y2sis 491 46 Wit9nessed4T6est 34 11 1385 30 Pu1b8lic Revi3e0w 50 3 3480 27
Resp.
33 35
32 — 94 23 1 84 71 8 83 33 — 82 46 14 69 25 4 56 58 10 72 55 23 66 45 5 88 14 — 86 45 3 65 59 14 73 41 — 44 48 — 42 34 2 41 48 15 40 59 22 41 34 11 35 50 3 40 24 11 37
4984 29 884 12 283 44 1842 25 3609 41 1536 19 792 14 1646 36 5808 36 2806 32 65 22 2773 32 1474 36 2492 34 1431 25 470 12
38 27 24 11 37
Bolded figures represent the category with the highest frequency
cfaotreegaocryh wasisthurtahnecehitgehchesntiqfrueq.uency
e.P, practitioner (also known as Junior); P(W), practitioner with su-
spJeurnviosiro);nP; (SW, s),enpiroarc;tiPtri,opnreirncwipitahl. su-
Table 2 – Number of people required.
ber of people (%) Total
Table 2 – Number of people required.
Assurance technique Num
Assurance technique Number of people (%) Total 1
1 2 3 4+ Resp. Review of [...] PPP 54
Reviewof[...]PPP
[...] Self Assessment Form Architectural Review Configuration Review Source Code Review Observation
Interview
Red Team Exercises Penetration Tests Vulnerability Scan
Social Engineering
Threat Assessment
[...5]4SelfA3ss8essme6ntForm1 9841
1 884 — 8462 9 6753 3 735
Arc8h1itectu1r3al Revi4ew Co7n0figura2ti0on Rev6iew
So6u3rce Co2d8e Revie5w
Ob4s9ervatio29n 12 10 694 Int6e4rview30 4 2 5464 Red44Team4E8xercise8s — 7110 Pen1e0tratio3n1Tests25 33 6278
Vu2ln8erabi5li9ty Sca1n1 So8ci4al Eng1i4neering2 Th4re2at As4s5essmen5t Sta7t3ic Ana2l1ysis 4
2 8740 4 863 4 8429
Static Analysis
Dynamic Analysis
Fuzzing
Formal Verification Cryptographic Validation Emanation Security Analysis Witnessed Test
Public Review
Dy7n5amic A18nalysis5
Fu6zz9ing 19 7
For7m8al Ve1r7ificatio—n
Cry4p0togra3p8hic Va1l0idation13 4506 Em5a6nation29Securi7ty Analy7sis 4610 Wi6tn0essed34Test 6 — 355 Pu5b5lic Rev3i3ew 13 — 409
49 27 5 19 37 Bolded figures represent the category
2 4649 5 4728 5 410
Bolded figures represent the cfaotreegaocryh wasisthurtahnecehitgehchesntiqfrueq.uency for each assurance technique.
38 6 1 94
13 Rev4iew of [2...] PPP84
20 [...]6Self As4sessme8n3t Form
28 Arc5hitectu4ral Revi8e2w
29 Co1n2figura1t0ion Rev6ie9w
30 Sou4rce Cod2e Revie5w6
48 Obs8ervati—on 71 31 In2t5erview33 67 59 Re1d1Team 1Exercise8s8 14 Pen2etratio—n Tests 86 45 Vul5nerabil9ity Scan65 21 Soc4ial Engi3neering73
18 Thr5eat Ass2essmen4t4
19 Sta7tic Anal5ysis 42
17 Dy—namic A5nalysis41
38 Fu1z0zing 13 40
29 For7mal Ver7ificatio4n1
34 Cry6ptogra—phic Val3id5ation
33 Em13anatio—n Securi4ty0 Analysis 27 Wit5nesse1d9Test 37
Public Review
with the highest frequency
Bolded figures represent the
for each assurance techniqu
P, practitioner (also known a
A significant difference was identified for Witnessed Test in A significant difference was identified for Witnessed Test in
Table 3 – Expertise requi Assurance technique
2 3 4+ Resp.
pervision; S, senior; Pr, principal.
the perceptions of Security Practitioners (M = 1.48, SD = 0.71,
the perceptions of Security Practitioners (M = 1.48, SD = 0.71, pared to when supervision is not provided. This has implications
N = 33) and Information Security Managers (M = 2, SD = 0.53,
N = 33) and Information Security Managers (M = 2, SD = 0.53, pared to when supervision ifsonroitspcrovsitd-efdf.eTchtivsehnaessismapslictactoionnsstitutes time from two roles.
N = 8); t(14) = −2.28, p = 0.039, two tailed. This difference may
N = 8); t(14) = −2.28, p = 0.039, two tailed. This difference may for its cost-effectiveness asFoitrcPornascttiittuiotensetrismwe iftrhoomutwsuopreorlevsis. ion, the dominant assur-
be a consequence of the lack of Security Practitioner famil-
be a consequence of the lack of Security Practitioner famil- For Practitioners without asnucpeetrevcihsinoinq,utehsewdeoremRienvaienwt aofsCsulier-nt-Completed Self-Assessment
iarity with Witnessed Test usage, which is largely constrained
iarity with Witnessed Test usage, which is largely constrained ance techniques were RevieFworomf Cs,liVenutl-nCeorambpiliettyedScSaenlf,-AasnsdesPsumbelnict Review.
to the compliance assessment process (e.g., for standards and
to the compliance assessment process (e.g., for standards and Forms, Vulnerability Scan, and PNuoblaicssRuevraienwc.e techniques were found to have a statisti- regulations). In this case, perceptions of Information Security
regulations). In this case, perceptions of Information Security No assurance techniquceasllywseigrneififocuantddtifofehreanvceea(ps