DSN 2007
The 37^th Annual IEEE/IFIP International Conference on Dependable Systems and Networks

Workshop on Assurance Cases for Security - The Metrics Challenge

Motivation and Theme

For critical systems it is important to know whether the system is trustworthy and to be able to communicate, review and debate the level of trust achieved. In the safety domain, explicit Safety Cases are increasingly required by law, regulations and standards. It has become common for the case to be made using a goal-based approach, where claims (or goals) are made about the system and arguments and evidence are presented to support those claims.

The need to understand risks is not just a safety issue: more and more organizations need to know their risks and to be able to communicate and address them to multiple stakeholders. The type of argumentation used for safety cases is not specific to safety alone, but it can be used to justify the adequacy of systems with respect to other attributes of interest including security, reliability, etc.

An international community has begun to form around this issue of generalized assurance cases and the challenge of moving from the rhetoric to the reality of being able to implement convincing and valid cases. In a recent article in IEEE Security and Privacy (http://www.computer.org/portal/pages/security/2006/v4n3/oth.xml) we outline what we have been doing so far in the security area, what we hope to achieve and where we go next.

Prior workshops, beginning with one held at DSN 2004, have identified a number of technical, policy and research challenges. This workshop will focus on one of these challenges: metrics for assurance cases for security. Such metrics can be essential for supporting decisions regarding the resources provided to develop the assurance case, and the efficacy of the resulting case. However, there is no commonly accepted approach to this topic. We would like to be able to answer questions (in the context of security) such as:

1. What makes an argument compelling?
2. Are there standard patterns for arguments?
3. What arguments should be compelling? What arguments do people actually find compelling?
4. How do additional arguments or evidence serve to increase the compelling nature of a case?
5. If there are accepted notions of what makes a case compelling, to what extent do we know that these accepted notions are correct or incorrect?
6. Is there a measure of compellingness that could be used to compare alternative argumentation structures?
7. How can assurance cases be composed? If they are composed, is it also possible to compose the metrics associated with the individual cases?
8. How can arguments with different compelling force be compounded for supporting the case claims?
9. What new types of evidence are needed to create arguments which are more sound and how will we measure that they are more sound?
10. By what metrics do we assess the effectiveness of evidence?
11. What is the cost/benefit justification for developing an assurance case?
12. Are there different levels of effort depending on the motivation? Can these levels be quantified?
13. Can it be shown that a well-defined and executed assurance case process will cost less than current assurance processes?
14. Given two cases, one that costs more and, by some metric, is more compelling than the other, how does one make the trade?

The purpose of the workshop is to understand these and other questions in the context of assurance cases for security and to identify viable technical approaches.

The workshop will be held on day two, June 27, of DSN 2007. It will consist of:

1. invited talks at the beginning of a session followed by brief presentations of position papers.
2. discussion of the application of metrics to example toy assurance cases for security.
3. consolidation and conclusions.

Objectives and Topics

The workshop will identify state of the practice in metrics for assurance cases in the context of security, identify promising ways forward and research directions. The workshop will produce the following outputs:

1. Identification of the candidate metrics for assurance cases for security and the characteristics which those metrics must posses
2. A listing of the major classes of evidence for assurance cases for security and a mapping of classes of evidence to metrics
3. Candidate methods for combining the various classes of evidence toward the desired system security properties.

Participation, Submission and Selection Process

Attendance at the workshop will be open to all interested parties. For active participation submission of a position paper of no more than six pages is required. The submission should conform to the proceedings publication format for IEEE Conferences and should be submitted electronically in PDF format via e-mail to weinstock[at]sei.cmu.edu. Please use the subject "DSN AC Workshop Submission" so that your submission is not overlooked. Submissions will be reviewed by the organizers and those accepted will be published in the DSN Proceedings supplemental volume.

Workshop Organizers

Robin Bloomfield, Center for Software Reliability (UK)
Marcelo Masera, Joint Research Center of the European Commission (Italy)
Ann Miller, University of Missouri at Rolla (US)
O. Sami Saydjari, Cyber Defense Agency (US)
Charles B. Weinstock, Software Engineering Institute (US)

Questions

For questions or concerns regarding the workshop please contact Sami Saydjari at the Cyber Defense Agency or Chuck Weinstock at the Software Engineering Institute.

Important Dates

Submission deadline:

Author notification:

Camera ready copy: