| &Associates |
Responder is really a platform for developing and doing network analysis and manipulation funtions. Because it is a general purpose programmable system, you can program it to do anything you want, but of course you then have to write and debug all of those programs. As a platform, it provides some reduction in programming effort for network-related wirk by providing the real-time interface to raw packets and lots of functions and macros that pull apart packets, transform them in useful ways, ans generally help get the more technical parts of the work done. This frees you - the programmer - to spend your time doing higher level things.
As a programmer, the folks who developed Responder used it to do some interesting things for their own interest and for special folks who have special needs. Some of these applications provide useful funtions for other applications as well. This part of the manual describes these Responder applications.
Responder has been used as an application platform to build an authoritative-only Domain Name Server (DNS). This was intended to augment and eventually replace the previous secure DNS server created by students at Sandia National Laboratories and to allow the folks running sites that use Responder to replace the more vulnerable servers they use today with less vulnerable servers that can also perform other functions,
Within the Responder control file, DNS responses are implemented by putting in the following lines:
(load "DNS.l") h DNSservers 1.2.3.4 4.5.6.7 ; provide IP addresses you want to provide DNS service from :(dnsreply) eth0 U * @DNSservers:53 - I ; DNS responses
That was easy... now for the hard part. You have to provide the information used for DNS lookups. This is done in a file called "dns.hosts". Here is an example:
(IN-PACKAGE "RAWSOCK") (defun gendnsentry (ips names &aux i) (loop for i in ips do (setf (gethash (list :PTR (format () "~d.~d.~d.~d.in-addr.arpa." (fourth i) (third i) (second i) (first i))) DNShash) names)) (loop for i in names do (setf (gethash (list :A i) DNShash) ips)) (loop for i in names do (setf (gethash (list :CNAME i) DNShash) names)) )
That part of the file has to be left alone. The following part of the file is to be adjusted to or augment4ed as you see fit. The first entry indicates that addresses 1.2.3.4, 5.6.7.8, and 9.8.2.3 should return both test.my.com and dns2.my.com as responses, and vica versa. The next line associates 7.5.6.5 with my.com. Any other requests for A records ending in ".my.com" will return 7.6.5.4. MX records for my.com will point to mail.my.com and MX records for any other machine name ending in .my.com will return my.com. CNAME and NS record defauls are also set in later lines as is an entery for localhost and my.com.
(gendnsentry (list '(1 2 3 4) '(5 6 7 8) '(9 8 2 3)) (list "test.my.com." "dns2.my.com.")) (gendnsentry (list '(7 5 6 5)) (list "my.com.")) (setf (gethash (list :DEFAULT :A ".my.com.") DNShash) (list '(7 6 5 4))) (setf (gethash (list :MX "my.com.") DNShash) (list 1 "mail.my.com.")) (setf (gethash (list :DEFAULT :MX ".my.com.") DNShash) (list 2 "my.com.")) (setf (gethash (list :A "localhost.localdomain.") DNShash) (list '(127 0 0 1))) (setf (gethash (list :CNAME "my.com.") DNShash) (list "my.com.")) (setf (gethash (list :DEFAULT :CNAME ".my.com.") DNShash) (list "my.com.")) (setf (gethash (list :DEFAULT :NS ".my.com.") DNShash) (list 0 "my.com."))After the file is set up, you simply run the router with 'router-run responses' or whatever the proper response file is, and DNS services will be provided. This is particularly useful in cases where a DNS server is down. You can pop up a replacement server for that DNS on another existing machine running Responder while repairing the older machine, and the network will never know. In some cases, we provide backups for many DNS servers in one Responder so that whenever one goes down, Responder can serve the DNS till they return. With a little bit of programming, you can even make this process automatic. One responder can back up a whole network.
The Network Intelligence Tool (NIT) is a Responder application. It uses Responder as a platform to implement a wide variety of other desirable functions similar to other tools that gather information from networks, with a few differences.
Difference 1: NIT emits on packets. Not ARP packets, not error packets - no packets. It is completely passive, so that when you come up on a White Glove, you can run NIT without assigning IP addresses, and it will gather information from traffic going past on the network without detection. In fact, you can hook up several network cables to different networks and it will listen to all of the content from all of the networks and analyze it.
Difference 2: NIT figures out things that other tools don't figure out. For example, NIT will find user IDs and passwords sent over the network and store then for your use. NIT will also find things like remote NAT firewalls, and determine what operating systems are running on remote machines as they enter your network, and things like that.
Difference 3: NIT allows you to change what it does and how it does it. For example, you can have it find all packets with a particular pattern in them and save them - or perhaps you prefer to save all packet content from subsequent packets between those IP addresses - or perhaps jsut form that session. Any of these are easily done with NIT. Suppose you wanted to do something stranger - like use the detection of a string to trigger other string detections, or to start sending all of the packets from that particular source IP address out a separate port where you had other analysis tools? No problem. NIT is a completely programmable network intelligence tool.
Difference 4: Here's my last example - but hardly the last difference. Suppose you wanted to use NIT as an intrusion detection tool - perhaps by taking SNORT IDS specifications and running them in Responder. No problem. You can even write a SNORT to Responder translator to take the entire SNORT database and turn it into Responder detection sequences. And here's the really good part. You aren't limited to waiting for later to respond to these attacks. You can program Responder to act on them right away if you want to - and it can react in any way you want it to.
OK - so by now, you get the idea. A typical NIT response file is contained in NIT-responses. An example is shown here. IT starts by notifying the user of what it is. Then it loads various program elements that are part of its operation. The basic intelligence package, followed by string handling, passive OS fingerprinting, response supporting software, DNS unpacker, and the regular expressipon parser.
(format t "~%********************Welcome to NIT *****************~%") ;; standard intel stuff (load "intel.lsp") ;; string functions for simplistic stuff (load "stringthing.lsp") ;; fingerprints (load "finger.lsp") (read-fpfile) ;;smtp and other responses for the game - and supporting stuff (load "smtp.lsp") ;;DNS listener (can also respond) (load "DNS.l") ;; regular expression parser and analyzer for content analysis (ext::cd #P"regex/") (load "load.lsp") (ext::cd #P"../")
Next, we see the interpretation of inputs. Any UDP packets from or to a port 53 are interpreted as DNS packets with content analyzed and stored. Next we do operating system fingerprinting using passive fingerprints. Then we add TCP, ICMP, and UDP mac address and IP address storage and TTL storage. Then ARP storage, TCP port information of various sorts, and so forth.
:(getdnsheader) * U *:53 * c I ; DNS thing :(getdnsheader) * U * *:53 c I ; DNS thing :(fingerprint) * T * * cS I ; fingerprint SYNs :(additem "SRC:" (frommac) (src) t) :(additem "DST:" (tomac) (dst) t) :(additem "TTL:" (src) (list (ttl)) t) * TIU * * c I ; show it all and continue :(additem "ARP:" (frommac) (tomac) t) * A * * c I ; show it all and continue :(additem "TDP:" (dst) (list (tcpdstport))) * T * *:0-1024 c I ; show it all and continue :(additem "TDP:" (dst) (list (tcpdstport))) * T * *:8080 c I ; show it all and continue :(additem "TSP:" (src) (list (tcpsrcport))) * T *:0-1024 * c I ; show it all and continue :(additem "TSP:" (src) (list (tcpsrcport))) * T *:8080 * c I ; show it all and continue :(additem "USP:" (src) (list (tcpsrcport))) * U *:0-1024 * c I ; show it all and continue :(additem "UDP:" (dst) (list (tcpdstport))) * U * *:0-1024 c I ; show it all and continue :(countitem "U:" (src) (tcpsrcport) (dst) (tcpdstport)) * U * * c I ; show it all and continue
This next snippet i sused to find user IDs and passwords, and if we uncomment the first line it will search for content with either Fred or Cohen (exact spelling) in any TCP packet with data enclosed and store it as part of the content hash. It saves TCP header information in another hash, counts a few more things, and that is it.
;;:(if (findtokens (tcpcontents) (list "Cohen" "Fred")) (setf (gethash (tcpcontents) contenthash) (list (dst) (tcpdstport) (tcpsrcport)))) :(findusers) * T * * cP I ; Find users and passwords on Push TCP packets :(savetcpheaders) :(countitem "T:" (src) (tcpsrcport) (dst) (tcpdstport)) * T * * c I ; show it all and continue :(countitem "I:" (src) (icmptype) (dst) (icmptype)) * I * * c I ; show it all and continue .
Of course collecting and storing all of this information is only part of what NIT does. Many of these also display novel information on the user console - which can be filtered to produce selective output if desired. But NIT is best seen through the perl GUI called face.pl - which is run from X11 by typing into an X window as follows:
The interface provides an effective way to select different items of interest and view them.
If you are going to be able to test out your Responder, you will also need to be able to generate any desired sequence of bits and send them over a network toward any desired interface of the Responder. This is the purpose of the traffic generator (TG).
Of course if all you want to do is put bit sequences on the Ethernet, all you have to do is fill the output buffer with those bytes like this:
(loop for i from 0 to SNAPLEN do (setf (b i) 0)) (loop for i from 0 below (length "eth0") do (setf (element (slot from 'SA_DATA) i) (nth i "eth0"))) (sendto socket SNAPLEN)
That fills the byte array with zeros, specifies the ethernet for the outbound transmission, and sends the data out. If you want to send something different, you will have to fill in the details. That's where TG comes in. It help to fill in the details so you don't have to work with one byte at a time and figure out where to put everything. Here is an extract from the comment section of the tg.lsp file:
For convenience a number of constants are defined:
macto = 0 - position of (6 byte) to-address in mac header
macfrom = 6 - position of (6 byte) from-address in mac header
mactype = 12 - position of (2 byte) type in mac header
default-ip an argument to :initial-contents that initializes
mactype = #x800 (= IP)
IP version = 4
IHL = 5
TOS = 0
total length = 40
id = 0
flags = fragment offset = 0
ttl = #x40
protocol = 6 (tcp)
IP-version = 14 - position of version, IHL
IP-tos = 15 - position of tos
IP-length = 16 - position of (2 byte) total length
IP-id = 18 - position of (2 byte) id
IP-frag = 20 - position of (2 byte) fragment flags and offset
IP-ttl = 22 - position of ttl
IP-proto = 23 - position of protocol
IP-checksum = 24 - position of (2 byte) checksum
IP-source = 26 - position of (4 byte) source address
IP-dest = 30 - position of (4 byte) dest address
IP-options = 34 - position of IP options (variable length)
All of these tcp constants assume zero length IP options.
You can, of course use (+ tcp-source (offset)) to get the proper
location within a real packet.
TCP-source = 34 - position of (2 byte) source port
TCP-dest = 36 - position of (2 byte) dest port
TCP-seq = 38 - position of (4 byte) sequence #
TCP-ack = 42 - position of (4 byte) acknowledgment #
TCP-offset = 46 - left half of this byte is length of tcp header
TCP-bits = 47 - right 6 bits are urg,ack,psh,rst,syn,fin
TCP-window = 48 - position of (2 byte) window
TCP-checksum = 50 - position of (2 byte) checksum
TCP-urg = 52 - position of (2 byte) urgent pointer
TCP-options = 54 - position of TCP options (variable length)
These definitions help build desired packets up. For example, if I wanted to create a packet rigged from MAC address 1:2:3:4:5:6 to mac address 9:8:7:6:5:4 from IP address 10.11.12.13 to IP address 20.19.18.17, I might fill in the packet like this:
(for i from 0 to 6 do (setf (b (+ macfrom i)) (nth i '(1 2 3 4 5 6)))) (for i from 0 to 6 do (setf (b (+ macto i)) (nth i '(9 8 7 6 5 4)))) (for i from 0 to 4 do (setf (b (+ ip-source i)) (nth i '(10 11 12 13)))) (for i from 0 to 4 do (setf (b (+ ip-dest i)) (nth i '(20 19 18 17))))
To further ease the burden, a funtion called sendpackets sets up defaults sensible for TCP packets. Here is a good example of hos sendpackets can be used to create a packet for transmission.
(sendpackets :device "eth0" :bytes (list macfrom 6 #x00e0291cb909) :bytes (list macto 6 #x00045a417537) :nbytes 54 ;; minimal tcp packet :initialize default-ip :ipchecksum t ;; a good idea for most IP packets :bytes (list ip-id 2 0 #xffff) ;; random id :bytes (list ip-source 4 #x0a000102) :bytes (list ip-dest 4 #x0a000103) :tcpchecksum t ;; a good idea for most IP packets :bytes (list tcp-source 4 #x22223333) ;; from port 2222 to port 3333 :bytes (list tcp-seq 4 #x55555555) :bytes (list tcp-ack 4 #x66666666) :bytes (list tcp-offset 4 #x50107777) ;; offset, bits, window )
That particular example will send the described packet to the described interface. But sendpackets can do a little bit more than that. sending more than one packet. In particular, here is an extract from the TG comments:
Sendpackets accepts an arbitrary number of keyword arguments of the following forms. :frequencyhow many per second to send (default 1) :nsec how many seconds to send (default 1) :npackets how many packets to send (default 1) [if two of above are supplied then the other can be computed, if all three are supplied we ignore one] :nbytes number of bytes to send (default 100) :IPchecksum if t then compute ip header checksum (default nil) :TCPchecksum if t then compute tcp checksum (default nil) :UDPchecksum if t then compute udp checksum (default nil) :ICMPchecksum if t then compute icmp checksum (default nil) :device send on device of this name (default - whatever data was last used) :initialize (array from to) initialize the packet contents in positions to thru from (inclusive) to the contents of array in the same positions :bits (index nbits minvalue maxvalue) set the bits in the packet starting from to a random number between and , inclusive. All numbers are interpreted modulo that 2^ . If the first number is larger than the first (modulo the size) then the range wraps around. If the nbits is 4 then (3 5) or (19 5) means {3 4 5} whereas (15 17) or (15 1) means {15 0 1}. :bits (index nbits value) means same as :bits (index nbits value value) :bytes (index nbytes minvalue maxvalue) means same as :bits (<8*index> <8*nbytes> minvalue maxvalue) :bytes (index nbytes value) means same as :bytes (index nbytes value value)
This function also allows the sender to generate packets at a desired rate and send them out an interface. It is very handy for performance testing. One of th emore interesting tests we have used it for its to determine the optimal packet size for filling an Ethernet with packets. If the packets are too large, send failures result, while packets that are too small will not fill the Ethernet bandwidth.
In particular, the function 'findknee' repeatedly calls the function 'blitz', which sends the specified number of bytes from whatever last happened to be in the output buffer out the last specified interface, a specified number of times, with no delay between them, and reports the number of send failures.
By reviewing the output of findknee, you will clearly see the knee point - which is the optimal point for filling an Ethernet with packets. This is different on every ethernet card and each CPU because it is limited by performance characteristics of the devices.
Of course one of the best ways to use TG is to automatically generate test packets for verifying the operation of programs like SNORT, which was mentioned above. By taking the patterns provided in the SNORT library, TG can easily simulate all of the attacks that SNORT is supposed to handle, using any desired source and destination IP addresses and ports. This is a wonderful way to verify that SNORT responses do what they are supposed to do and to make sure that malicious attackers cannot exploit the SNORT configuration to attack your networks.
The Logger uses an additional lisp file called "Logger.lsp" to augment Responder for logging functions. It is specifically designed to allow Responder's selection mechanisms such as router-based packet selection and pattern matching to be applied to logging server applications. It also comes with a graphical interface for common usage.
The added functions come in two types:
File control funtions: These functions allow log files to be open'ed, flush'ed, and close'd so that the programmer can have control over filenames used for logging each of the logfiles generated by the Logger. Typically, log files are associated with each functional type of log being generated. For example, a different logfile might be generated for a packet log associated with each of a set of ongoing investigations, and a different file would be associated with SYSLOG entries gathered from a variety of servers. Each file can be controlled when necessary and appropriate to the function.
loginit: (loginit 'handlename "pathname") opens or creates th enamed logfile and associates it with the identified handle for append-only access.
logclose: (logclose 'handlename) closes the identified logfile and writes out any pending output.
logflush: (logflush 'handlename "pathname") closes and reopens the logfile, thus forcing output to be flushed while keeping the file usable. Unflushed logs will be flushed by default every 4K bytes of output on a typical system.
Log generation funtions: These functions append log data to logfiles using standard logging formats. For example, CSV file formats are provided for linkage to external databases, while text-based files might be desired to simulate syslog outputs. This list of standard functions are provided in source form so that they can be copied to build new customized logger formats as desired for the specific situation.
syslog: (syslog 'filehandle [format]) produces a standard syslog output to the previously openned file using the specified format (:csv or :txt) and places "SYSLOG" as the first item in each line of output. Output is one line per entry of lenth as long as required. This function replaces special characters that are unprintable or violate the applicable standard with a coded version of the character. Output includes Date and time stamp, interface of arrival, source and destination MAC, IP, and port addresses, and the content portion of the packet.
tcplog: (tcplog 'filehandle [format] [coment]) produces a standard tcp log output to the previously openned file using the specified format (:csv or :txt) and places the provided comment (specified in double quotes e.g., "25") in the first entry of each log line. Output is one line per entry of lenth as long as required. This function replaces special characters that are unprintable or violate the applicable standard with a coded version of the character. Output includes Date and time stamp, interface of arrival, source and destination MAC, IP, and port addresses, and the content portion of the packet.
udplog: (udplog 'filehandle [format] [coment]) produces a standard udp log output to the previously openned file using the specified format (:csv or :txt) and places the provided comment (specified in double quotes e.g., "53") in the first entry of each log line. Output is one line per entry of lenth as long as required. This function replaces special characters that are unprintable or violate the applicable standard with a coded version of the character. Output includes Date and time stamp, interface of arrival, source and destination MAC, IP, and port addresses, and the content portion of the packet.
fulllog: (fulllog 'filehandle [format] [coment]) produces a standard full log output to the previously openned file using the specified format (:csv or :txt) and places the provided comment (specified in double quotes e.g., "WATCHLIST-3") in the first entry of each log line. Output is one line per entry of lenth as long as required. This function replaces special characters that are unprintable or violate the applicable standard with a coded version of the character. Output includes Date and time stamp, interface of arrival, and all bytes of the packet.
Logger also relies heavily on Responder functions, the Responder router-file format, and built-in lisp functions:
Formatting functions: These funtions are built into the Responder tool and demonstrated in the Logging server. They are provided to allow complex formats to be generated so that any desired output can be produced for external application use. For example, you might want data formatted for direct access by an external database server that can not import from CSV files.
Packet selection funtions: This is done largely by the router syntax of Responder and is described there.
Lisp Capabilities: The lisp capabilities configured with Responder include string search capabilities which are very helpful in investigations. For example, finite state machine implementations allow Responder to be programmed to start logging based on previous pattern matches.
When you combine these into a Responder, you get a very flexible logging server with excellent performance, covert operation, and full programmability.