Security Assessments, Evaluations, and Audits, and ROI
Qu: How do you know you are secure? The answer is you must conduct assessments and evaluations to know. Being secure doesn’t mean your system can’t be hacked or compromised. There is always a risk! (“Prevention eventually fails.”) To be secure means you have an acceptable level of risk.
Assessment
A Security assessment is the first step. The assessment involves the whole organization and assesses the non-technical aspects of security. It is useless (or worse, leading to a false sense of security) to evaluate the technical aspects before you know what must be protected. For example, evaluating the network encryption of data is pointless if the data is easily obtained from a non-protected database.
The assessment should examine the security policies (and related policies, such as disaster recovery policy or DRP), procedures, organizational structure and infrastructure architecture.
However a preliminary step is to determine exactly what resources are considered vital to protect. You must know what the organization considers valuable and/or private. If possible dollar amounts should be assigned. (This step is related to the risk analysis done for a DRP. Note you can often get help assigning dollar amounts from organization’s finance/insurance dept.)
Next you need to make sure the policies, procedures, organization, and infrastructure, if all implemented correctly, would protect that data to the levels required. Often, an organization will adopt inappropriate policies and procedures (sometimes self-contradictory ones). It doesn’t matter how well you have implemented security measures if they wouldn’t protect the data that needs protecting!
You must also make sure the policies and procedures are in compliance with applicable laws, regulations, and best practices. Examples include HIPAA (Health Insurance Portability and Account Act),SOX (Sabanes-Oxley), GLB (Gramm-Leach-Bliley), DMCA, FMA (Financial Management and Accountability act), FERPA (Family Education Rights and Privacy Act), CFAA (Computer Fraud and Abuse Act), and many other federal and state regulations and laws.
Return on
Investment (ROI) [Adopted from Network Security Evaluation
by Cunningham et. al,
(C)2005 Syngress, p. 57]
A quality assessment of a medium to large sized organization could take months for a (small) team. For this reason such assessments (like risk analysis) are usually left to consultants who will receive tens of thousands of dollars for the work. To justify this expense the system admin should be able to explain the current and expected return on investment (ROI) or ROSI (return on security investment) for the assessment and/or evaluation. (And indeed, for any security related budget.)
Unfortunately there isn’t a single widely agreed upon method for this calculation. (Do a Google search for “security ROI”.) Traditional ROI calculates an actual payback and/or reduction in expenses. But for security money spent you will rarely if ever see such a return. Instead you must focus on security expenses as a kind of necessary insurance, and the ROI is the lowered risk exposure.
Understanding Risk [from: www.csoonline.com/read/120902/calculate.html]
According to one study the American Society of Safety Engineers (ASSE) cites, the ROI of fire extinguishers is in fact about a $3 return for every $1 invested if you take fire extinguishers as part of a larger corporate health and safety initiative—which you should, since fire extinguishers (like IT security) rarely show up as a discrete security purchase.
... A binary thinker might suggest that, since there was no fire last year, there was no ROI. If that is the attitude at your company, it's time to initiate some awareness and education because that's not how risk mitigation works. Think of it this way: If you wear your seat belt but don't get in a car accident, does that mean you ought not invest in a seat belt because there was no return?
No. You did get a return, because return is not measured in a dogmatic world of what did or did not occur, but in the stochastic world of what might occur and how likely it is to occur. That is the game of risk; prepare for something to happen by investing in ways to stop it from happening.
Calculating ROI
One way the security ROI can be estimated is by computing the annual estimated loss expectancy of a security incident, which can be calculated as:
Incident-cost x probability x mitigation + cost-of-mitigation
Note a mitigation is some measure taken that either reduces the probability of an incident, or reduces the cost of recovery should some incident occur. The annual estimated loss expectancy is the sum of these calculations for each incident, for some combination of mitigations.
Suppose the cost of a security break-in $1,000,000 and the probability of this is 35% if you do nothing. Then the annual loss expectancy if you take no action is $1,000,000 x 0.35 = $350,000.
Next consider what mitigations you can take. Each has a cost so you need to determine which mitigations are the most cost-effective, given the limitations of your overall budget.
Suppose a $20,000 anti-virus program might reduce the chance of occurrence by 50%, a $10,000 user security awareness training program might reduce the expected loss by (say) 50%, and a $15,000 security assessment and evaluation by 35%. The choices are:
|
Mitigation |
Cost |
% Improvmnt |
Expected
annual loss |
ROI |
|
None |
0 |
0 |
350,0000 |
0 |
|
Anti-virus |
20,000 |
50% |
195,000 |
155,000 |
|
Training |
10,000 |
50% |
245,000 |
105,000 |
|
Assess & Eval |
15,000 |
30% |
242,500 |
107,500 |
Of course you need some reliable industry data to back up these claims for improvement. Also, you can combine various mitigations to reduce the expected loss (and hence increase the ROI). Also in a real calculation you would need to consider initial or one-time (capital) costs (and depreciation) and recurring expenses. Qu: if your annual budget was $25,000, what mitigations would you recommend?
“Building security into software engineering at the design stage nets a 21% ROSI. Waiting until the implementation stage reduces that to 15%. At the testing stage, the ROSI falls to 12 percent.” – from: www.cio.com/archive/021502/security.html
For the loss that is left after all mitigations, you can get cyber-insurance. Some sources include: Lloyd’s e-comprehensive, Chubb’s Cypersecurity, and AIG’s Net Advantage Security.
The NSA (National Security Agency) has published the Information Assurance Methodology (IAM), with practical and very valuable advice on how to conduct an assessment, including for example how to negotiate the contract for the consultants. (Show book.)
Evaluation
Once you have determined the correct procedures, policies, and infrastructure required to provide adequate protection, it is time to conduct a Security Evaluation. In this phase you examine the equipment and configurations to make sure your routers, switches, firewalls, servers, and hosts will implements the policies determined by the assessment.
The NSA has also published the Information Evaluation Methodology (IEM) as a guide to conducting the evaluation. (Show book.)
The definitions of assessments and evaluations as explained here are not universally agreed upon terms. Many would consider these terms interchangeable with each other, and also with an audit.
Audits
The main goal of IT audits, assessments, and evaluations is to reduce business risk and liability exposure. IT has become a critical part of an overall business strategy. The difference between assessments, evaluations, and audits are not necessarily in how they are conducted but rather in the goals and intended audience.
There are many types of audits, even just considering IT audits. Once type of audit is a routine part of on-going security measures (This can be called an assessment/evaluation). Another type is performed after a problem has been discovered (or suspected), to assess what is wrong with the policies and procedures and to assign blame.
While an organization may conduct assessments and evaluations to ensure to themselves (internal audit or assessment) the system works correctly and efficiently, audits are often conducted to provide the same assurances to shareholders, business partners, customers, and government regulators (external audit).
Auditors ensure:
· An adequate audit trail exists for all activities, especially financial transactions.
· Controls to ensure data integrity.
· Both unit (subsystem) and integration (whole system) testing procedures, to make sure the system performs as intended.
· Exceptional conditions are anticipated as much as possible, and general exception handling procedures are in place. This includes authorization procedures for system overrides when needed.
· Controls exits to prevent unauthorized and/or undocumented changes to the infrastructure (and indeed, to any procedures).
· Required government, industry, and internal policies and procedures are followed. (Sometimes referred to as compliance audits.)
· Adequate training is provided for personnel.
· Additional security, compatibility (different vendor equipment), ..., controls.