Over the last several years, computing has changed to an almost purely networked environment, but the technical aspects of information protection have not kept up. As a result, the success of information security programs have increasingly become a function of our ability to make prudent management decisions about organizational activities. Managing Network Security takes a management view of protection and seeks to reconcile the need for security with the limitations of technology.
When I started using game theory and strategic gaming at Sandia to help analyze and improve understanding of information protection decisions, the first thing I told my boss was that, contrary to government and corporate policy and regulations, I was going to start using government computers to play games. He told me that I should make certain that the graphical interface didn't look too slick - but to also make sure it looked slick enough - and advised me to change the word GAME on the screen to another word - like ANALYSIS or PLANNING or SIMULATION - all more acceptable ways to express what has, over the years, become a widely accepted and commonly used technique of framing the issues surrounding complex decision-making.
The movie WAR GAMES really popularized the strategic gaming activities used by military organizations for several hundred years to help understand strategy and tactics, but in the United States government and elsewhere throughout the world, strategic game theory really come into widespread use following the mathematical analysis of game theory by John von Neumann in the 1940s. By now, the basics of zero-sum game theory are taught in management schools as a part of the fundamental understanding of optimization. The field of operations research concentrates largely on the solution to mathematical programming problems that are mathematically akin to game theory. In short, whether we know it by that name or not, we are often analyzing games when we seek to find effective solutions to strategic and tactical problems in organizations.
Applying games to info-sec is a relatively recent phenomena. In fact, the first information war games started less than ten years ago. The reason we started to use these games to look at information warfare was because the challenges we faced became to complex to analyze with any other technique - and because strategic gaming is ideally suited to investigating a new field of inquiry with high complexity and diverse sources of knowledge. The use of game theory in network protection is even more recent than in information warfare. The first use of games to analyze network protection, as far as I am aware, started in the last few years.
Three different kinds of games have remained in widespread use after many years of experimenting with gaming. They are, roughly, strategic games, automatic games, and simulations for exploring a game space. The rest of this month's article will be oriented toward describing these games and how they have been used or are coming to be used in network protection management.
Strategic games are, in essence, directed discussions which use a set of players with an appropriate mix of backgrounds and knowledge to explore issues surrounding some long-term decisions. They are most often played by creating a scenario in which the players are put into roles and forced to make decisions in a constrained amount of time.
The objectives of most strategic games include:
From a process point of view, a typical strategic game is played as a set of moves. Several common game types are used as examples here to get a sense of how these moves are used:
If there are two major limitations of strategic games, they are context boundedness and the substitution of breadth for depth in problem analysis. Context boundedness is simply a result of forcing scenarios on the players - the more specific and constraining the scenarios - the more limited are the results - and the more applicable the results are to very similar situations. Context boundedness can be eleviated to a large extent by adaptive scenarios and live referees, but this is quite difficult to do in the short time frames usually reserved for games. The substitution of breadth for depth is also a side effect of the short time usually provided for games. Too little time is provided for ideas to germinate during these games, again with the exception of games played over longer time frames.
The best examples that I have been involved with addressing network protection challenges in strategic games were played at the national infrastructure levels. In these games, national level policies are examined relative to protection of critical infrastructure under a range of scenarios. In the worst case, global conflicts between the United States and peer or near-peer competitors are the setting and considerable limitations are placed on the ability to track down the sources of increasingly malicious and potentially harmful attacks. The last such game I was involved in (as of this writing) was in December of 1996. I have designed and implemented similar, but more limited games, played with corporate consultants and information protection specialists.
While strategic games provide insight into strategies, the more common gaming method used in network protection is the automatic game. In an automatic game, combinations of offensive and defensive strategies are characterized by payoff functions for attackers and defenders. The automatic game is then, in essence, a solution generator that searches a solution space to provides optimal strategy mixes for offensive and defensive players.
Automatic games of this sort of sometimes presented as mathematical programming problems and optimized using operations research techniques. In the case of simple games or selections among a small number of strategies, automatic games are useful, but substantial challenges arise as the number of strategies grows. For example, for a game with 100 attacks strategies and 100 defensive strategies, there are 10,000 entries in the payoff matrix that have to be filled in.
If the game is a zero-sum game, where that attacker gains what the defender loses and vica versa, min-max techniques lead to a rapid solution, sometimes providing insight into mixed strategic approaches. For non-zero-sum games, strategic analysis can degenerate into large tree searches that are beyond current computational capabilities for large strategy sets.
Most real-world strategies in the network attack and defense realm involve sequences of moves with learning by both parties. In these cases, the tree structures become even more complex because the set of all histories may have to be considered in determining an optimal strategy.
Fortunately, in many cases, the goal is not an optimal strategy, but rather one that satisfies some management criteria. For example, management may require only that the return on investment for defensive strategies be as much or more than the return on investment required for other corporate investments. In this case, the game only needs to be played until a strategy satisfying that requirement is found. From then on, improvements may be helpful, but are not required, and computation can be spent on an as available basis. IT is not guaranteed that any solution will be found that meets the criteria, but it is quite common to find several strategies that do, and it is often possible to quickly eliminate many strategies that will not fulfill the criteria, yielding a more tractable problem space.
The real challenge in automatic gaming for network protection analysis is in creating a set of attack and defense strategies that are realistic, and assessing payoffs that make sense.
If a detailed analysis is attempted, this is pretty similar to the techniques used in risk analysis and, as last month's article on risk analysis and risk management points out, there are serious limitations to this technique. Fortunately, game theory allows us to explore game spaces with less certain information. In fact, in many automatic games, we can get the same strategic decisions by simply ranking attacks and defenses in order of effectiveness and playing the game to solve for the best min-max of rankings. The results for optimal strategies will be the same except in cases of mixed strategies where the ratios of the strategies will be artificial. If a maxed strategy results, it will be necessary to perform a more detailed analysis to determine the proper ratios of different strategies for optimal effectiveness.
Getting the strategies to use in playing automatic games is a more complex problem, and one that is often solved by using strategic games. A set of strategic games are played to generate a set of strategies. Than analysis is done to rank the attack and defense strategies in terms of their relative payoffs against each other - perhaps using a 1-10 scale or a poor/adequate/good/great scale, etc. and optionally including management views on the suitability of the strategies to the organization. Finally, the automatic game is run to find an optimal strategy mix.
The results of the automatic game are normally fed back to the participants as one of the inputs to the decision process. It is rare that automatic game results are used blindly to make strategic protection decisions.
In cases where the space is too large for automatic games to be played in reasonable amounts of time, or where the generation of the matrix is too complex, error prone, or time consuming to be practical, simulations may provide a method for exploring the game space.
Building a meaningful simulation is a fairly complex task. It requires that you develop a model which is accurate enough to be useful but simple enough to be simulated in a reasonable amount of time. Then you have to program the model, provide all of the initial data required for its operation, and provide a means for trying each sequence of moves. A pseudo-random number generator is required, and it must be of a sort that does not bias the paths through the space - the one that comes with your computer is almost never the right kind. A means for getting results out in a meaningful way is also required, and that typically involves a graphical interface or analysis capability.
The purpose of simulations such as this are to explore the game space, perhaps with the intent of finding a path through the space that is within tolerable limits. Depending on what and how you simulate, it may be difficult to get meaningful data out of the simulation. For example, some techniques tell you how well a particular run did, but don't allow you to recreate the run in detail to show how the results were derived and support the results with the sequence that resulted in them.
In many cases, it's unclear whether the simulation tells you about the model, the simulator, or the thing you are trying to understand. This is because of context boundedness - the limited view of reality given by the simulator is bound to give results related to the view - which may be substantially different than the thing being modeled.
The runs through a complex simulation may also do little to explore a significant sampling of the overall simulation space. The resulting lack of spatial characterization means that the results seen in the simulation reflect only a small subset of the possibilities and may ignore enormous ranges of possibilities.
Complex interactions may not model reality well, particularly those that are sequence dependent. For example, in 1984 when trying to simulate computer viruses spreading through computer systems at a detailed level, we found that the precise sequence of running programs resulted in enormous differences in outcomes. In fact, rather than converging, our simulation runs diverged. This is a characteristic of many sequential problems typical of complex interactions.
The good part of simulations is that they are fun to do. This means that people will likely use them. Many people claim to learn from such games, and I have had learning experiences in my gaming. In many cases, games provide a fairly realistic picture that may yield understanding and be explanatory to people who don't initially believe the outcomes are possible.
In 1985, I developed what I believe to be the first Internet-based strategic game. The game was actually played out by about 30 participants of the information warfare mailing list and concentrated on sequences of events in the 5-20 year time frame within the United States. Development of the scenario by people who were already experts in the subject matter took about 5 person-days of effort. Creating, testing, and preparing the programs for playing the game took another week. The game itself ran over the period of about a week, and the analysis took another few days. This game produced a lot of results related to how email-based communications worked in strategic planning, issues related to secrecy vs. integrity, how people build or don't build teams electronically, and of course the players learned a lot about themselves and future issues.
An automatic game was recently developed using a Web-based interface and applied to analyzing strategies for protection of computer networks. The approach taken in this particular game was to derive results from return on investment for both attacker and defender. The creation of the automation took only a few days, but assessing the investment and return values for different scenarios took several person-weeks of effort for a relatively small network. Once the figures are in place, the analysis takes only a few seconds, with the results painted on the screen using colors to indicate strengths and weaknesses. The results were easily understood by management - something rarely attained by computer-generated data.
A simulation research effort was recently undertaken to try to simulate attack and defense in substantial computer networks. The intent is to analyze more than a hundred classes of attacks and defenses distributed throughout a large network in an effort to increase the minimum cost of attack. The mathematical modeling and analysis are still underway and the effort will likely take more than a person-year before meaningful results become available.
While hand-held video-games can be purchased for only a few dollars, serious games that explore protection issues for a corporation are far more expensive.
A typical strategic game costs from US$40,00 to US$100,000 to develop and play the first time. Thereafter, the game can be replayed several times for only $10,000 to $20,000 per play, depending on the details of the game and what is to be derived from it. I have seen strategic games that cost as much as $1,000,000 to create, play once, and generate a report.
Most of the cost of automatic games is driven by providing the data used by the game to do its analysis. It may cost as little as $10,000 to generate a program that can implement a particular type of game, but providing the details used by the game for its analysis ultimately drives the return value and takes the lion's share of the effort. For a substantial game reflective of a decision made for a billion-dollar corporation, costs in the range of $50,000 to $100,000 are typical. Once a custom automatic game is developed, it can be replayed with slightly different parameters relatively quickly and without substantial added costs. This playing of scenarios is most helpful in analyzing decisions over time and often justifies the cost of the game.
Simulations can range in cost from tens of thousands of dollars to hundreds of millions of dollars depending on what is being simulated. A typical simulation for network protection analysis costs on the order of $50,000 to develop and $50,000 to populate with data. Simulations of this sort may be run many times at little added cost.
What people say about games varies widely. Some executives think that strategic games are the most valuable experiences they ever have in understanding the issues surrounding a key decision - while others say that they are a waste of time. It's best to present it as a learning experience the first time out - if they like it, do it some more - if they don't, you've learned not to do it again. There is less variation of opinion in automatic games and simulations, where most people agree that they model something and are occasionally useful. Automatic games are typically used by analysts as a tool to support generating reports on key decisions, and this may account for the lack of diversity in assessed value. Simulations are almost universally accepted.
Several companies offer services to those who would like to develop games, and mine is one of them, so you'll have to find the others on your own. I do feel confident in telling you that it is far better to pay what it costs for a game developed by people who do gaming for a living than to try to build your own game or buy it from the least expensive vendor that claims to be able to get the job done. A rule of thumb may help here:
Gaming can help you do slightly better in making high-valued decisions and quickly help people understand complex issues. If the cost of doing the game right is not justified by the incremental value of a better decision or the better understanding of the complex issue, you shouldn't be using a game to help you decide.
Fred Cohen is a Senior Member of Technical Staff at Sandia National Laboratories and a Senior Partner of Fred Cohen and Associates in Livermore California, an executive consulting and education group specializing information protection. He can be reached by sending email to fred at all.net.