On the Brittleness of Software and the Infeasibility of Security Metrics How secure is a computer system? Bridges have a load limit; this isn't determined (as "Calvin and Hobbes" would have it) by building an identical bridge and running trucks over it until it collapses. In a more relevant vein, safes are rated for how long they'll resist attack under given circumstances. Can we do the same for software? I've sometimes quoted Lord Kelvin: ``If you can not measure it, you can not improve it.'' ``When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of *Science*, whatever the matter may be.'' But I've reluctantly concluded that current architectures are not amenable to metrics of the sort I want. Here's why. It's well known that any single defense can fail. More precisely, it's well known that any piece of software can be buggy, including security software -- that list is alarmingly long. This means that whatever the defense, a single well-placed blow can shatter it. We can layer defenses, but once a layer is broken the next layer is exposed; it, of course, has the same problem. Karger and Schell noted more than 30 years ago the difficulty of defending against an attacker who had purchased a copy of your system and could probe each layer offline. In most situations, we need to defend against threats of precisely this nature. And this is why metrics are so hard: the attacker's effort is linear in the number of layers, rather than exponential, and each effort is low-cost. What we need is defense systems with this exponential property. That is, we need a system where getting through two layers is proportional to the product of the difficulty of each layer, rather than the sum. If our defense systems do have this property, we have some hope of measuring their strength. The constant for any one layer may remain small -- this is, after all, software that we're dealing with -- but we've somehow found a compositional principle that negates the linear effect. Unfortunately, we do not know how to do this. One possibility might be to use randomization techniques to increase the attack constant for a particular layer. Unfortunately, we're still dealing with software, and the attacker may go around our randomization. Thus, we can use random stack frame layouts, similar to OpenBSD's, to defend against buffer overflows; the attacker, though, could launch an SQL injection attack. Perhaps we could use intrusion detection and repair at each layer. If we can do that, the holes won't stay open at first; the attacker will have continually relaunch the attack. That presupposes that we can build such self-repairing software -- research on it is still at a very early stage -- and that it won't be subject to the same brittleness. In any event, the repair would have to succeed before the next layer was penetrated, or some autonomous attack code could continue the attack on inner layers even though the outer layers had been repaired. The problem, of course, is that such systems do not exist today. The strength of each layer approximates zero; adding these together doesn't help. We need layers of *assured* strength; we don't have them. I thus very reluctantly conclude that security metrics are chimeras for the foreseeable future. We can develop probabilities of vulnerability, based on things like Microsoft's Relative Attack Surface Quotient, the effort expended in code audits, and the like, but we cannot measure strength until we overcome brittleness. And until we can measure security, we can't improve it.