Networks dominate today's computing landscape and commercial technical protection is lagging behind attack technology. As a result, protection program success depends more on prudent management decisions than on the selection of technical safeguards. Managing Network Security takes a management view of protection and seeks to reconcile the need for security with the limitations of technology.
I am always trying to improve my firewalling techniques. While I realize that firewalls are - with rare exceptions - basically leaky sieves, I still try to figure out how to allow the proper access without leaving the gates wide open. Recently, I have been trying to tighten down some of the specs on what gets in and out, and I thought I would share my windows experience with you.
The first thing that shows up when you boot Windows, before the login screen - before the user-installed applications start up, is this nifty piece of work:
... > 224.0.0.2: icmp: type-#10
This little audit record shows that Windows tries to reach "ALL-ROUTERS.MCAST.NET" - in other words, it wants to talk to multicast routers. Now I didn't ask Windows to talk to multicast routers - or in fact - to do any service whatsoever involving multicast. But I am getting ahead of myself...
I should start from the beginning. I run various firewalls, more or less for the fun and convenience (huh) of it. As an example, my kids go out on the Internet over our cable modem, and I do lots of Internet-based work myself as does my wife. So naturally, I want to protect my family from all the bad things on the net.
Perhaps even more importantly, I want to balance security with utility. For example, I want to be able to do backups of the kids computer without having to buy a tape drive for it, so I back up to a file server which is periodically backed up to tape or disk or whatever. I also want some reasonable amount of privacy, like for example, I don't want outsiders from the Internet looking over my family pictures - and I don't want to get constantly solicited, and I don't want outsiders to be able to corrupt or destroy my family pictures or the work that my wife and I are doing. The problem is, how do I do it?
I do a few things that many corporations are unwilling to do - but probably should do. I control configurations to the point where (for example) javascript is disabled on browsers. I don't allow windows ports to interact with the Internet. I have a set of internal IP addresses that don't exist on the Internet, so any cross-over will not route unless it comes through the address translation mechanism of the firewall. I run internal DNSs - for internal addresses only, I have some network segments that are isolated from others, and I do a lot of monitoring. I don't share users on workstations (except the kids who share a single Internet box), I do comprehensive backups on a regular basis and store copies where they should be kept, I have UPSs in place for critical systems, I have redundancy at several levels, and on and on.
But even with all of this in place and more, I still have the same problem almost every other infrastructure in the world has. Every once in a while, in order to get the job done, we need to load software from the Internet and run it on an internal machine. At the same time, those same machines that have this external software have to connect to the Internet - at least for Web access - in order to do their jobs - and they need to use DNS to look up IP addresses by host names. There's just no getting around it in the environment we are required to operate in.
I find myself having the same conversation again and again with potential clients. They tell me about some security product and how they want me to evaluate it and I tell them that my first likely attack will be to get some user to load my software and control their system through DNS or HTTP traffic initiated at their side. If they understand me, they choke for a minute, tell me thank you, and don't call again for a long time - if ever. If they don't understand, they are very polite, express that they may call me to do the work - but never do. Whether it's denial or ignorance (I'm sure I just blew any chance of working for any of them...) the answer is the same. The way we use the Internet has inherent risks, and no magic technology will save us from them.
Now, of course, I say that and at the same time push more and more technology at reducing my risks. So what technology can I use to reduce the risks associated with the commercial products that insist on phoning home using common ports? Especially in light of the ease with which somebody can take out a DNS server or web site and place their own site in their place to gain remote control over my systems?
My solution is to watch closely. More and more closely at higher and higher bandwidth for more and more hours I may add. I do this in a variety of ways. Here are some of them:
I test things out before I put them on the net. For example, when I install new software, I install it on a test system first and watch for network traffic under various conditions. The test system is generally rigged to limit traffic and is isolated from other parts of my environment, but looks to itself like it's on the Internet. As I watch the traffic, I notice various things that I don't like - in particular things that - as the user - I don't explicitly tell it to do - and I block them out at the firewall. As I block them, I notice the fall back positions they take and block them. Round and round we go and where it stops... nobody knows. Eventually, I figure I have them all, and I release them for real use.
Then, I watch some more. On a regular basis, I audit behaviors of systems and traffic. That means that I take complete network logs for periods of time, watch for encrypted content, watch for larger than expected packets, watch for packets when there are no users, and so forth.
Then, I watch some more. I also do random checks of system traffic and behavior - both from the systems themselves and from other systems that monitor silently at different points in my infrastructures. I review various audit trails - particularly in firewalls and critical systems.
Then, I watch some more. I also investigate what I find. For example, earlier today, I got a strange notice:
Apr 16 05:20:48 all login: 2 LOGIN FAILURES ON tty1, 192.168.17.63
It turns out a user had pasted the IP address into the login prompt mistakenly - but for a while, I figured someone from the Internet with a forged (and illegal) IP address had somehow bypassed a filter and managed to cause an attempted login to the console even though they were clearly not physically present! I investigate all of the internal anomalies down to resolution.
I do other sorts of watching, and am always seeking to improve my techniques, but I think you get the picture. Watching helps a lot, but it's only a start.
Some time back, I decided that I wasn't going to be able to stop all attempts at exfiltrating data. Even though I have been very successful to date, some day somebody will find one of the holes that I have to leave in order to provide functionality and start to exploit it. I have done it to myself, so I know it's possible in practice - not just in theory. I also know that no matter how much I watch, there is always a way to get it through - unnoticed and unstopped for at least long enough to get some meaningful data exfiltrated.
But I don't give up. My goal is to stay ahead - at least a few steps ahead. As part of this goal, I try to anticipate what the bad guys will do next and after than and after that, and provide prevention, detection, and response for each of those three steps ahead that I try to stay. In this way I have a tendency to detect when they catch up by a step and I can extend my actions another step ahead. I do this largely through deceptions of one form or another.
Of course I do. In a variety of ways. Here are some of them:
I have lots of information, only a small portion of which is important to me. This means that it is unlikely that they will stumble across anything important very quickly, so it gives me more time to detect them before harm is likely to be done.
I have phony systems, addresses, services, files, and so forth strewn all over the place. These are inexpensive and provide an early warning against attempts to do intelligence. In some cases, I even let people break into these phony systems to watch what they do.
I have a marking system that allows me to detect information going where it doesn't belong. Since exfiltration typically takes a few steps, I have a chance to detect the markings before the data is actually released or used.
I use integrity controls to detect unauthorized modifications. Some are real-time, while some are random and periodic. Some are off-the-shelf, and some are custom. This mix provides added assurance against any single point of failure.
Between all of this auditing and analysis and deception and so forth, I still have time every once in a while to do some actual work... which brings me back to the story of the month:
After blocking the ports associated with conducent.com, and after a few new reboots, lo and behold, my Windows box adapted...
08:06:38.990000 216.33.199.88.23 > ...: . ack 315 win 8447 (DF) ...
Now, instead of going to conducent.com on port 80 or 8080, the system reverted to port 23 (a telnet port) on a different server. If you look this one up, it is a name server at goibsdsl.com, but on http requests to port 23 it reports out as part of conducent's network. Once it comes up, it starts exchanging the same details of my system as the previous connections did. Then came 216.33.210.71 port 23. Because I am intolerant of such things, I blocked all port 23 outbound from the Windows box. So now, ports 80, 8080, and 23 were blocked as well as ICMP and DNS from the windows box to unauthorized places. Then the 216.32.73 address range comes up (extensis.com) - this time on port 17027 - this program is quite adaptive - so I blocked the class C associated with that network - and while I was at it - all recipient ports greater than 1024. At this point, the program took 15-20 minutes to get to the next thing it would try, so I set the logging up for long-term observation.
All of this effort for just one piece of software may seem a bit extreme, but in an enterprise environment, this sort of testing provides improved understanding of what to look for in your detection system as well as how to prevent this data exfiltration.
The vendor in this case was Computer Associates and the product was InoculateIT. Once I really shut it down, it provided a pop-up screen indicating that it could not update itself and indicating how it could be manually updated. The product has an auto-update feature, but it cannot be disabled. You can configure all sorts of things about it, but you can't tell it to stop trying. Even though I have all checking turned off, it still automatically runs to try to update itself. Now just to be fair, I contacted the vendor to see what they thought I should do. Here's my request:
I would like to know the precise firewall rules I should use to allow normal usage while blocking all updates and other traffic related to your product (InoculateIT) that is not generated by explicit user requests. I would also like to know in detail what information is exchanged between your products at my site and your site, how this information is used, and what of it is stored, transfered, or otherwise handled.
Their answer... will be posted here as soon as I get one...
From my view, the deal is pretty big. For example, part of my security posture, and part of my company confidential information, is how I configure my firewall and how my internal network functions. If somebody wants to send in a Trojan horse, it will make it far easier for them to go undetected if they can count on a vendor to go through different sequences of exfiltration attempts. These folks are, in my view, illicitly exfiltrating trade secrets that will grant them and others the ability to bypass my controls. They are able to tell what internal IP addresses I use, what my internal servers are and what protocols they are running, and they have done an intelligence process to determine what ports are open for further exfiltration. This method also leads to a greater ability to defeat my deceptions, since it provides a cross check on other information that users gain from the intelligence efforts that I fend off with deceptions.
Just in case you think all of this is not worth doing, you should know that within a few hours of cutting off messages from one of the sites, I got a probe on the NetBus port (12345) from 38.31.117.30 - a public dial-in access point to psi.com's network. In the probe, one of the sites I visited during my investigation of this product popped up. Whether that probe is the result of my cutting off the signals or of my looking at their web site during my investigation is unclear, but what is clear is that this is not an accident. Take these infiltrations seriously!
I know that when I do red teaming, this sort of information is incredibly valuable for getting in and getting results and control out. It's so much the worse when I find this in security products. Want another good example? How about a product like 'RealSecure', which advertises that it is supposed to be really secure. From outside your network, I can tell when you are doing firewall administration, and the coded messages it sends back over the Internet... I bet you don't know what information they are exfiltrating.
These products are only the tip of the iceberg. Increasingly, the Internet is the means for products to communicate back to their makers and get updates from their makers. This ultimately means that your security depends on their security, and their security depends on their vendors' security, and so forth. It means that I can often sit outside your network and tell when you are in and when you are out, what you are doing, information about your configuration and how to exploit it - and so much more. And this is just from your security products!
As somebody who make decisions about security and manages and advises others about their security decisions, I have to say that trying to chase down every one of these Trojan horses is pretty draining. I am, on occasion, tempted to just give it up and trust anybody and everybody who wants to send me software and exploit my network. But in the end, I am just too stubborn or some such thing to give it up. I know that there are risks and benefits to be weighed, but I guess that the one thing I demand out of my vendors - especially my security vendors - is disclosure of what new vulnerabilities I am introducing when I choose their products.
I generally advise my clients to know what their networks are supposed to be doing and check them to see that they are doing nothing else. But with the introduction of these undocumented and intentionally hard-to-control mechanisms, this task becomes monumental. The security testing process is, of course, critical to any organization, but at some point we need to save ourselves the individual testing and group together to do community testing and open reporting of such things. I've now done my part on one or two products in this month's article. It's time to do your part.
Here's the part where I do something really stupid and volunteer. If you send me the details of what products do what of these sorts of things, I will make sure that you get on the list of people who get the reports from everyone else. I'm going to do it all via a mailing list if enough of you contact me and want to get involved. For now - email to sectests@egroups.com
About The Author:
Fred Cohen is exploring the minimum raise as a Principal Member of Technical Staff at Sandia National Laboratories, helping clients meet their information protection needs as the Managing Director of Fred Cohen and Associates in Livermore California, and educating defenders over-the-Internet on all aspects of information protection as a practitioner in residence in the University of New Haven's Forensic Sciences Program. He can be reached by sending email to fred at all.net or visiting /