Tools

Digital forensic evidence is usually latent, in that it can only be seen by the trier of fact at the desired level of detail through the use of tools. In order for tools to be properly applied to a legal standard, it is normally required that the people who use these tools properly apply their scientific knowledge, skill, experience, training, and/or education to use a methodology that is reliable to within defined standards, to show the history, pedigree, and reliability of the tools, proper testing and calibration of those tools, and their application to functions they are reliable at performing within the limitations of their reliable application.

Tools are used in all phases of evidence processing. In order for tools used in forensic processes to be accepted by the legal system, the tools have to be properly applied by people who know how to use them properly following a methodology that meets the legal requirements associated with the particular jurisdiction. (FRE 701-706) [3]

Methodology: One of the key things that experts need to know about is the tools that they use. This is because tools are used in almost all tasks associated with DFE processing and tool failures that yield wrong results or tool output that is not properly interpreted leads to opinions and conclusions that may be wrong. One of the main tasks of the DFE expert witness is to identify a meaningful methodology for applying tools to address the legal issues and use that methodology and tools that implement it with known accuracy and precision by examining the evidence and the claims made with regard to the evidence. While some of the claims may be understood with only the experts knowledge, such as assertions that are inconsistent with each other or that fly in the face of current scientific thinking in the field of expertise, most claims in legal matters that involve DFE involve the application of scientific methodologies to evidence through tools.

Pedigree: and History: Tools have history and pedigree that helps to indicate their reliability. Depending on the extent to which the tool provides scientific results that are not obviously verifiable by independent means by others, these factors are more important or less important. For example, if a tool, such as the Unix command "wc" counts the number of words, lines, and characters in a file, and the result is used to draw a conclusion about the evidence in the matter, it is something that can be readily confirmed or refuted by any party by simply counting, or in the case of files with many lines, using an independent tool. In this case, the history and pedigree are less important than that the tool has shown reliability at the task it is being relied upon to carry out, that it has been adequately tested, and that it be properly calibrated for its intended use.

Reliability: and Testing: While testing of tools may be reasonably done by those who have background in testing of digital systems or by independent bodies, such as NIST, which performs select test of forensic tools in the United States [12], calibration must be done by the digital forensics expert prior to and after the use of the tool, assuming that that is required for validation of the tool's accuracy and precision to the level being used for presentation of the results of its use. Very little testing has been formalized in this field for the specific needs of digital forensics, so examiners wishing to be prudent should undertake their own testing programs, and this should be a normal part of the process used in preparing for legal matters where such tools are used. There is a substantial body of well defined knowledge in testing of digital systems, including refereed professional journals, books, conferences, and classes at the undergraduate and graduate level. As an example, the IEEE has had a refereed journal on the subject since 1984. [13]

Testing of tools is fundamental to their use, and in the field of DFE, an individual brought forth as an expert who has not tested their tools and does not know their function and limitations in adequate detail, is unlikely to be able to withstand cross-examination with regard to those tools or the things those tools are being applied to. This may, ultimately, lead to their disqualification as an expert, or the disregarding of their testimony as not meeting the standards required for credible expert testimony.

Calibration: The notion of calibration is foreign to many in the digital computer arena, largely because, unlike analog devices which have minor variances due to temperature, pressure, and other physical conditions, digital systems, when working within normal operating ranges, produce either 1s or 0s and do so with very high reliability. Nevertheless, there are calibrations that can and should be done prior to and after the use of DFE tools to validate that what was done did not introduce inaccuracies into the process. As an example, when doing a forensic image of digital media to a different media, the destination media should be pre-configured to a known state so that process failures can be detected. Otherwise, residual data from previous events or from the manufacturing process might be mistakenly intermixed with the new DFE to produce corrupted results. This sort of spoliation has the potential to create enormous problems if the tools and media are not properly calibrated, if error messages are not carefully preserved and taken into account, if contemporaneous logs of the forensic activities are not produced and retained, and if evidence isn't created to verify that the image taken is a true copy of the original evidence. This is similar to the process of cleaning a pipet for a chemical analysis, testing the cleaned pipet to verify that it is free of contaminants, processing the sample, getting the result, then verifying that the pipet is free of contaminants after the sample is analyzed. Failure to undertake such a process would violate standard procedure in chemical testing that has been shown to produce faulty chemical analysis. Similarly, failure to undertake measures to calibrate and verify digital forensic processing of evidence can introduce contaminants or produce faulty digital analysis.

Digital forensic analysis processes often include the creation of special purpose filters, the development of search criteria, and the authoring of small computer programs, sometimes including combinations of scripts written in languages such as the command language of the Unix shell, the Perl language, and other programs written in other languages, and pre-packaged utility programs that come with systems, such as the stream editor "sed", the regular expression string search program "grep", and many other similar sorts of elements. These are commonly combined with tools that retrieve data from Internet sites and process them in various ways to produce outputs that show some analytical result.

Function: and Limitations: When such tools produce results that are readily verified by inspection, such as counts of how many lines of particular types were at particular locations within particular files, the conclusions themselves constitute a testable result that the opposition can challenge and verify. As such, the tools and techniques need not be shown; however, when introducing such evidence, it is incumbent on the producing party to make certain that the results are accurate and precise. To the extent that they are in error and the opposition can demonstrate this, the court will often levy sanctions and potentially exclude the expert and the results from use in court under the admissibility restriction that the results are less probative than prejudicial, the expert witness is not reliably applying a scientific method to the evidence, and that the expert is not in fact adequately knowledgeable or skilled to express scientific opinions to the trier of fact. It is incumbent on experts to provide details of the limits of their results in terms of the limits of accuracy and precision and to not overstate results. For example, when analyzing text files against a format specification, the expert had better understand the extent to which the formal specification is reflected in actual use, and examine results produced for anomalies before declaring the results of the program to be precise and accurate. To the extent that anomalies are detected, they should be explained and the precision and accuracy of results properly characterized.