MEMORANDUM AND ORDER RE: MOTION TO EXCLUDE EXPERT TESTIMONY

GERTNER, District Judge.

I. INTRODUCTION

The government has charged defendant Rudy Frabizio ("Frabizio") with possession of child pornography under the Child Pornography Prevention Act ("CPPA"), 18 U.S.C. § 2252(a)(4)(B). To obtain a conviction under § 2252(a)(4)(B), the Supreme Court has held that the government must prove the defendant possessed images of real children, rather than images of virtual children, engaging in sexually explicit conduct. Ashcroft v. Free Speech Coalition, 535 U.S. 234 (2002); United States v. Hilton, 386 F.3d 13, 18 (1st Cir. 2004). The task is a difficult one, becoming more difficult by the minute, as digital imaging techniques become increasingly sophisticated.

To meet its burden of proof, the government seeks to introduce the testimony of Thomas Musheno ("Musheno"), a forensic examiner of photographic evidence in the FBI's Forensic Audio, Video, and Image Analysis Unit. Specifically, the government offers testimony that, after visually examining the photographs Frabizio allegedly possessed, and without using any specialized equipment, Musheno concluded that six of the nineteen images definitely depict real children and ten others "appear to be" real children.

Frabizio moves [docket entry # 87] to exclude the proffered testimony under Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), on the grounds that Musheno's technique is unreliable because it has not been tested, has not been subjected to independent peer review, has an unknown error rate, has no real standards or controls, has not achieved general acceptance, and satisfies no other set of reasonable reliability criteria. See Daubert, 509 U.S. at 593-94; Kumho Tire Co., Ltd. v. Carmichael, 526 U.S. 137, 158 (1999). Accordingly, I held a hearing to determine the admissibility of Musheno's proffered testimony on May 5, 6, and 11, 2005. See Daubert, 509 U.S. 579, 113 S.Ct.2786, 125 L.Ed.2d 469.

After the hearing, however, I noted a far more fundamental issue lurking beneath the questions about Musheno's particular methodology or error rate. However skilled he may be, he bases his testimony on observations of the photographs. The threshold question is whether visual observation is at all appropriate to the task at hand: distinguishing real images from virtual ones.

If it is possible to distinguish the real from the virtual with the naked eye, then the specialized observational skills of aDaubert-qualified photograph expert could help the jurors make their own observations of the evidence. As such, the photograph expert's testimony might be admissible, assuming Daubert's other requirements were satisfied. See United States v. Shay, 57 F.3d 126, 132 (1st Cir. 1995) (noting that Federal Rule of Evidence 702 requires that, to be admissible, expert testimony must "assist the trier of fact").

On the other hand, if visual observation, even by a seasoned observer, cannot distinguish real and computer-generated images in this case, then observation alone fails to address the threshold question and is therefore irrelevant. The testimony of a photograph expert would be inadmissible; an individual with expertise in computer-generated images would be required.

The First Circuit has left this issue open. In the instant case, I am obliged to resolve it. Based on the evidence presented to me about the current state of technology and the specific images involved here, I conclude that neither an expert witness nor a lay jury, using only visual means, can determine whether the images in this case are real or virtual to the level of certainty required in a criminal prosecution. I therefore find that Musheno's testimony is not helpful and must be excluded.II. THE STATE OF TECHNOLOGY AND VISUAL OBSERVATION OFIMAGES

Because the threshold question in determining the admissibility of Musheno's testimony is whether it is possible to create wholly computerized images that are visually indistinguishable from real photographs, my analysis begins here. This question raises both factual and legal issues. The factual inquiry involves the capabilities of modern computer programs as well as the particular images in the case at bar. The legal question concerns how much uncertainty about the efficacy of visual observation can be tolerated in a criminal case, where the burden of proof weighs heavily on the government.

After three days of hearings and multiple briefings on theDaubert issue, I have serious doubts as to whether a person visually studying the images in this case can distinguish real pictures from manipulated or wholly virtual ones with the level of confidence required in a criminal prosecution.

The Supreme Court acknowledged this difficulty two years ago inFree Speech Coalition, even as it held that the government must prove that the images are real:

[T]he Government says that the possibility of producing images by using computer imaging makes it very difficult for it to prosecute those who produce pornography by using real children. Experts, we are told, may have difficulty in saying whether the pictures were made by using real children or by using computer imaging. The necessary solution, the argument runs, is to prohibit both kinds of images. The argument, in essence, is that protected speech may be banned as a means to ban unprotected speech. This analysis turns the First Amendment upside down.

The Government may not suppress lawful speech as the means to suppress unlawful speech. Protected speech does not become unprotected merely because it resembles the latter.

Although several circuits have held that expert testimony and extrinsic evidence are not required to meet the government's burden of proof, the First Circuit is not among them. See, e.g., United States v. Irving, 432 F.3d 110, 27-28 (2d Cir. 2006) ("We decline to hold the government is required . . . to present expert testimony proving the children in the unlawful images are in fact real children. . . . [A] reasonable jury could conclude that the images depicted real children solely on the basis of the images themselves," but limiting this holding to cases involving video images); United States v. Kimler, 335 F.3d 1132, 1142 (10th Cir. 2003) cert. denied,Kimler v. United States, 540 U.S. 1083 (2003) ("Juries are still capable of distinguishing between real and virtual images."); United States v. Slanina, 359 F.3d 356, 357 (5th Cir. 2004).

When the First Circuit first visited this issue in United States v. Hilton, the court held that the "government must introduce relevant evidence in addition to the images to prove the children are real." 363 F.3d 58, 64 (1st Cir. 2004), vacated, 2004 U.S. App. LEXIS 19528 (1st Cir. 2004) (emphasis added). Further, the court acknowledged that the "vast technological revolution underway . . . has made undeniable the fact that sexually explicit images portraying children can be produced by artificial means." Id. at 64-65. More recently, the court again suggested that this question has yet to be resolved in this circuit:

In the First Circuit, it remains an open question whether, to prove the reality of an image, the government must present relevant evidence in addition to the image. The decisions of other circuit courts of appeals addressing this question are not binding in this court.

The government has previously conceded the difficulty of distinguishing real from virtual or manipulated images. In its brief in the Free Speech Coalition case before the Supreme Court, the government acknowledged Congressional findings on digital imagery:

Congress found that `new photographic and computer [imaging] technologies make it possible to produce * * * visual depictions of what appear to be children engaging in sexually explicit conduct that are virtually indistinguishable to the unsuspecting viewer from unretouched photographic images of actual children engaging in sexually explicit conduct.'

Even Musheno, the government's supposed-expert in this case, acknowledged that a trained artist might be able to create realistic-looking images. He testified: "I don't know if I've ever seen an image, a painting that is so photorealistic that it appears to be real, although is it possible to generate that? I guess it's possible." (Daubert Hr'g Tr. 20, 37-38, May 11, 2005.)

A significant body of literature also indicates that digitally manufactured images may be confused with real photographs. Faculty in the Department of Computer Science at Dartmouth College, for example, have noted "photorealistic images can be created that are nearly impossible to differentiate from photographic images." S. Lyu and H. Farid, "How Realistic is Photorealistic?" 53(2) IEEE Transactions on Signal Processing (2005) (available at http://www.cs.dartmouth.edu/farid/publications). Other articles suggest that such virtual image creation can be achieved using current technology and that even "experts cannot know whether a digital image is real or virtual." Timothy J. Perla, Attempting to End the Cycle of Virtual Pornography Prohibitions, 83 B.U. L.Rev. 1209, 1216 (2003). See also, A.C. Popescu and H. Farid, "Exposing Digital Forgeries by Detecting Traces of Re-Sampling," 53(2)IEEE Transactions on Signal Processing (2005) (available athttp://www.cs.dartmouth.edu/farid/publications) ("[D]igital images can be easily manipulated and altered. Digital forgeries, often leaving no visual clues of having been tampered with, can be indistinguishable from authentic photographs"); Caught On Camera, NEW SCIENTIST, Sept. 6, 2003 at 5 ("Warnings about the potential for faking digital images are not new. But the proliferation of cheap digital cameras and computers, together with programs for altering photos and editing video footage, is turning that potential into reality. Where once a specialist was needed to alter analogue images, even beginners can now create digital fakes good enough to fool discerning experts."). But see Susan S. Kreston, Defeating the Virtual Defense in Child Pornography Prosecutions, 4 J. High Tech. L. 49, 62 (2004) ("Creating realistic images of people . . . continues to be very difficult, with the difference between a real picture and one created by a computer, even using today's best technology, being discernable to the human eye.").

In addition, the defendant has submitted examples of wholly-computer-generated images that this Court finds to be nearly indistinguishable from real photographs. See, e.g., http://forums.cgsociety.org/showthread.php?t=141461; (depicting a nude woman in the fetal position);http://forums.cgsociety.org/showthread.php?t=361465 (showing a remarkable likeness of actress Jennifer Garner); see also http://forums.cgsociety.org/showthread.php?t=160012 (revealing a woman's nude back). Although this final image shows greater signs of being computer-generated than the other two, the fact that someone created an image of this quality in less than a week using Photoshop suggests the range of technological possibilities.

In this case, the government has not made the threshold showing that a visual observer can reliably evaluate the relevant pictures for signs of manipulation and computer-generation. Rather, the evidence strongly suggests that it is extremely difficult, if not impossible, for a photographic expert, let alone a lay observer, to determine whether the images involved in the instant case are real images or images created or manipulated through digital technology.

To the extent doubt remains on this threshold issue, I must resolve that doubt against the government. This is a criminal case. Defendant's liberty hangs in the balance, and the government bears the burden of proof.See United States v.Monteiro, 407 F. Supp. 2d 351, 356 (D. Mass. 2006) (noting in the Daubert context that the "burden of proof with respect to reliability remains on the proponent of the evidence"). Furthermore, as the First Circuit recently held, the defendant has no obligation to refute the reality of the images. Rather, it is the government's burden to establish that the pictures are real:

It bears repeating that the government is not released from its burden of proof by a defendant's failure to argue, or by an absence of evidence otherwise suggesting, the artificiality of the children portrayed. That the children in the images are real amounts to an element of the crime which the government must prove, the burden of which should not be displaced to the defendant as an affirmative defense.

I therefore conclude that visual observation is not adequate to the task of differentiating the real from the virtual in this case. My conclusion leads to two others: First, jurors cannot decide based on their own visual observations whether the images allegedly possessed by the defendant depict real children, as required by the Supreme Court's holding in Free Speech Coalition. Some expert testimony is required. Secondly, even a seasoned observer, such as an expert in photography, cannot make this distinction in the case at bar. Given current computer technology and the images involved here, visual observation, by either a jury or an expert, cannot identify which images depict real children and which have been manipulated or generated by computer. An expert with greater knowledge of computers must speak to that fundamental, threshold issue. III. MUSHENO'S PROFFERED TESTIMONY

As discussed above, I find that visual observation alone is inadequate to the task of evaluating the images in this case. If photographic experts as a general matter are inadequate to the task of identifying computer-generated images, then no level of experience in that field will suffice to qualify one as an expert. Indeed, allowing Musheno to testify would be like allowing a dentist to identify the causes of glaucoma.

If the government offered an expert who eliminated the possibility of such imagery in this case, then Musheno's testimony might be admissible. To address that possibility and to underscore the inadequacies of Musheno's technique as applied to the sophisticated task at hand, I will consider the application of Daubert to Musheno's proffered testimony.

Even if the government established that the images here were not wholly virtual, Musheno's methodology does not meet theDaubert standard. His technique amounts to little more than "eyeballing" the evidence and recording his conclusions on a checklist. Indeed, the Daubert discussion below underscores the limitations of Musheno's approach, especially with respect to sophisticated, downloaded computer graphics. A. Background

Thomas Musheno conducts forensic examinations of photographic evidence for the FBI's Forensic Audio, Video, and Image Analysis Unit ("FAVIAU"). Despite his experience with photographs, Musheno lacks an important piece of evidence that would substantiate his supposed expertise: proficiency testing. In addition, his computer training is decidedly dated. His most recent course on Photoshop, perhaps the dominant image manipulation software, was in 2000. (Daubert Hr'g Tr. 43, 45, May 6, 2005.)

Musheno's examinations can be broken down into four categories, including comparison analysis (for example, matching the image of a bank robber caught on tape and a person in custody); photogrammetry analysis (calculating measurements, such as a robber's height, based on photographs); and information extraction (enhancing an image to reveal additional details). (Daubert Hr'g Tr. 7-8, May 5, 2005.) The fourth category, the only one of Musheno's techniques directly relevant to this case and the area in which Musheno's experience is most limited, is image authentication: Musheno examines an image to determine whether it has been manipulated or digitally created from whole cloth. Id.

Musheno's image authentication analysis has several steps, each of which involves simply looking at the pictures. Upon receiving an allegedly pornographic image to examine, he first attempts to determine if the image depicts a known child victim. He compares the image with photographs from the National Center for Missing and Exploited Children database and images in his unit's Child Exploitation Obscenity Reference File ("CEORF") database.Id. at 19, 20. If the image matches a child in either database, Musheno ends his examination because there is no question that the image depicts a real child.Id.

Next, Musheno examines the image for artifacts consistent with manipulation, using what he called the "FAVIAU Single Image Evaluation Form." Id. at 21. However fancy the title, the form is nothing more than a checklist Musheno's unit created to provide a contemporaneous record of the examiner's observations. Daubert Hr'g, Ex. 1. The examiner records, for example, the media type in which the image was submitted; whether the image is in color, black and white, or monotone; and the size of the digital file. Id. at 22-23. He looks for compression artifacts, which are visible signs that a digital image has been condensed into a smaller file.Id. at 24.

The checklist requires the examiner to search for visible inconsistencies in seventeen listed categories, including the image's content, perspective, scale, shadows, quality of light, contrast, textures and patterns, sharpness, focus, and pixel values.Id. at 25, 27. An image would have inconsistent content, for example, if it depicted an elephant on a diving board and the diving board was not bent. Id. at 25. The examiner also inspects the image for rough edges that indicate digital cutting and pasting, as well as inconsistent grain structure and resolution.Id. at 27-28.

After noting any inconsistencies, the examiner considers fourteen listed human characteristics, asking a broad question — whether the image "look[s] human," looks "cartoonish," or falls somewhere in between. (Daubert Hr'g Tr. 10, May 6, 2005). This could not be a more subjective inquiry; it considers the subject's skin tones and textures, skeletal structure, and flesh and muscle movement to see what "appears to be real." Id. at 10-11.

If the image in question is part of a series of images, Musheno conducts additional analysis. He compares the facial and body features shown in the images, as well as the clothing and background detail, to determine if the same person is depicted in each image in the series. Id. at 16, 19, 26.

After completing the examination, Musheno drafts a report summarizing his observations and conclusions. Id. at 15-16, 20. There are four possible conclusions: that the person depicted is real, that the person "appears to be real but no conclusion" can be drawn, that the person is not real, or that no conclusion is possible. Id. at 20-21. An examiner might reach the "appears to be real" conclusion if the image appears to depict a real child but there is insufficient detail to reach a definitive conclusion. Id. No conclusion at all can be drawn, on the other hand, if, for example, multiple compressions have degraded the image. Id. at 21.

Significantly, Musheno conceded that it is possible to alter an image in a manner that is undetectable. He testified that because of this, he never opines that an image is real on the basis of a single image. Id. at 23. In such cases, he limits his conclusions to "appears to be real" and no conclusion. Id. at 24-25. He assigns the "real" label only if there are "multiple images and they're of great detail and they're complex and there are no artifacts consistent with manipulation or creation." Given the sophistication of this field, however, that approach is hardly adequate.

Musheno completes the authentication analysis by subjecting his conclusions to review, id. at 16, but this review is perfunctory. Rather than reviewing the images independently in a blind test, coming to his own conclusions and then comparing them to Musheno's, the reviewing examiner goes over Musheno's completed checklist, his report, and the images contemporaneously. Id. at 30-31. If the reviewer agrees with Musheno's conclusion, the process ends there. Id. at 31-32. If the reviewer disagrees, the reviewer and Musheno discuss the disagreement, which could lead to the drafting of a revised report that would also be reviewed. Id. at 32-33.

With regard to the images in this case, Musheno testified that he received an initial group of images for review in the summer of 2004. Id. at 34. He conducted the analysis outlined above, except that he did not even look for a series of images because it was not standard procedure at the time. Id. at 34-36. In approximately February 2005, he received additional images for review. Id. at 36. This time, he considered the images, along with images from another case, as a series. Id. at 37-38.

Musheno concluded that six of the images at issue depicted real children and another ten appeared to be real children. Another examiner confirmed Musheno's conclusions using only the review process described above. Id. at 38.

B. Daubert Analysis

As I have explained, I find that the government has not established that visual expertise, such as that of a photograph expert, can distinguish the real from the virtual in the face of modern computer technology, however, even if visual observation were sufficient, Musheno's methodology does not meet the reliability standards that the Supreme Court delineated inDaubert.1. Framework for the Analysis of Expert Testimony

The FBI has only been studying pictures of child pornography for signs of manipulation since the mid-to-late 1990s. (Daubert Hr'g Tr. 15, May 5, 2005.) In addition, the basis for the alleged expertise is constantly in flux because of rapidly changing computer technology. Thus, as with "new" scientific techniques, "the Daubert analysis is critical, and the court should be an especially vigilant gatekeeper."Hines, 55 F. Supp. 2d 62, 65 (D. Mass. 1999); United States v. Monteiro, 407 F. Supp. 2d 351, 358 (D. Mass. 2006) ("The Court's vigilant exercise of this gatekeeper role is critical because of the latitude given to expert witnesses to express their opinions on matters about which they have no firsthand knowledge, and because an expert's testimony may be given greater weight by the jury due to the expert's background and approach.").

Where, as here, the expert relies on his experience to justify his methodology, the Court must examine the experience underlying his conclusions. As the Advisory Committee noted in amending Federal Rule of Evidence 702, "no one denies that an expert might draw a conclusion from a set of observations based on extensive and specialized experience." Fed.R.Evid. 702 advisory committee's notes to 2000 Amendment (quoting Kumho Tire, 526 U.S. at 156). At the same time:

If the witness is relying solely or primarily on experience, then the witness must explain how that experience leads to the conclusion reached, why that experience is a sufficient basis for the opinion, and how that experience is reliably applied to the facts. The trial court's gatekeeping function requires more than simply `taking the expert's word for it.'

Specifically, Daubert outlined the following, non-exhaustive list of factors to consider in determining admissibility: 1) whether the expert's technique or theory can be or has been tested; 2) whether the expert's technique or theory has been subject to peer review and publication; 3) the known or potential error rate; 4) the existence and maintenance of standards and controls; and 5) whether the technique has been generally accepted in the relevant scientific community. 509 U.S. at 593-94.

Kumho Tire extended Daubert to non-traditional sciences, notably expertise based on observational, clinical, and technical judgments. 526 U.S. at 147, 149 (holding that the courts' "basic gatekeeping obligation" applies to "all expert testimony" and concluding that "[t]he trial judge's effort to assure that the specialized testimony is reliable and relevant can help the jury evaluate that foreign experience, whether the testimony reflects scientific, technical, or other specialized knowledge").

The Kumho Tire Court further instructed that the trial court may consider the Daubert factors in determining the admissibility of such testimony. Id. at 150, 119 S.Ct. 1167. The Court emphasized, however, that theDaubert factors are not a "definitive checklist."Id. (citing Daubert, 509 U.S. at 593), 113 S.Ct. 2786). The gatekeeping inquiry must be "tied to the facts of a particular case," and the law grants the trial judge "broad latitude" to identify the factors relevant to admissibility.Id. at 153, 150 (quoting Daubert, 509 U.S. at 591 (internal quotation omitted)).

I addressed the application of the Daubert/Kumho Tire framework to a technical field in Hines, where the government sought to introduce expert testimony about handwriting, and I concluded that handwriting analysis suffered a number of flaws.Hines, 55 F. Supp. 2d 62. It had never been subjected to meaningful validity testing or to financially disinterested peer review. I also took into account the fact that the testimony about similarities and differences in handwriting could be "understood and evaluated by the jury" and that the witness "can be cross examined." Id. at 70. Accordingly, I admitted the proffered expert to testify about his observations.

I reached the opposite conclusion with respect to the expert's conclusions. I noted that there were no studies of the technique's validity, no standards as to how many similarities it took to declare a match. Id. at 69. The error rate was unknown.Id. Unlike his observations, which could easily be understood and evaluated by the jury, the ultimate conclusion would be based only on the expert's experience — a leap that "may not at all be justified by the underlying data." Id. at 70. Accordingly, I excluded testimony about the expert's conclusions.

It is with this framework in mind that I examine Musheno's proffered testimony. 2. Application of the Daubert Factors

I will examine each of the factors delineated in Daubert before considering what other factors, if any, speak to the technique's reliability. a. Testing

The first inquiry under Daubert is whether the technique in question can be or has been tested. See Daubert, 509 U.S. at 593 (deeming testing a "key question"). Although observational fields such as image authentication need not be subjected to the laboratory tests that would be required of a traditional science,see Hines, 55 F. Supp. 2d at 68-69 (considering the lack of testing for proffered handwriting analysis testimony), the lack of testing here is startling.

During the Daubert hearing, I repeatedly asked Musheno whether his technique had ever been tested through, for instance, the use of blind testing, in which he would be asked to identify a variety of samples to determine whether they were real. The examiner, knowing which samples were, in fact, real and which were virtual, could then evaluate Musheno's performance. See (Daubert Hr'g Tr. 64, May 6, 2005). Musheno responded, "I've never been tested."Id.

Absent this type of proficiency testing, neither this Court nor the jury can assess the reliability of Musheno's work. This is particularly true where, as here, the field as a whole has no known error rates, making it impossible to guess how often Musheno is likely to be right or wrong. See Daubert, 43 F.3d at 1319 ("We've been presented with only the experts' qualifications, their conclusions and their assurances of reliability. Under Daubert, that's not enough."). Quite clearly, the first Daubert factor does not indicate that Musheno's technique is reliable.

b. Peer Review

The second Daubert factor considers peer review. The "peer review" process Musheno described leaves much to be desired.See (Daubert Hr'g Tr. 16, 30, May 6, 2005). Rather than conducting an independent examination of the images, Musheno's co-worker analyzed the images contemporaneously with Musheno's checklist and report, fully aware of Musheno's conclusions. Id. at 30. Musheno testified that no reviewer had ever disagreed with his conclusions — a result that could indicate either a flawless record or, equally likely, a review process that functions as a rubber stamp. Id. at 121-22. Indeed, the review process Musheno described runs a substantial risk of "examiner bias," a phenomenon by which an examiner who expects a particular result tends to find it. See William C. Thompson Michele Nethercott,The Challenge of Forensic Science, 28 Champion 50, 50 (Sept.-Oct. 2004) (observing that "[t]he failure of forensic labs to use `blind' procedures for interpreting test results contributes to the production of inaccurate conclusions in lab reports and courtroom testimony"). See also D. Michael Risinger et al., TheDaubert/Kumho Implications of Observer Effects in Forensic Science: Hidden Problems of Expectation and Suggestion, 90 Cal. L.Rev. 1 (2002).

Technical fields need not be held to the standard of peer review applicable to traditional sciences, which are often considered in scholarly journals. See Daubert, 509 U.S. at 593-94, 113 S.Ct. 2786. It is nevertheless appropriate to consider whether the fields have been subjected to meaningful peer review in determining its reliability. See Hines, 55 F. Supp. 2d at 68. The "peer review" conducted here is a far cry from the type of independent review that would bolster the technique's reliability.

c. Error Rate

The third Daubert factor concerns the known or potential error rate. This is perhaps the most problematic factor in this case, as the Court has no way of determining whether Musheno's years of experience add up to actual expertise. Although he has looked at many images in his career, no one knows whether his technique produces reliable results. Indeed, even if Musheno were the world's most talented assessor of image manipulation, this would mean little if the error rate in the field were particularly high. This problem is of particular concern where, as here, any expertise in the field of image manipulation would not address the possibility of an image that was not a real photograph manipulated by computer but rather was wholly digital. On the current record, there is no way of knowing whether Musheno's technique can recognize such computer-generated images. Lack of a known error rate also prevents the jury from assessing the proper level of deference to accord the expert's conclusions. Because Musheno's technique has never been tested, its error rate is unknown and therefore does not support a finding of reliability. See Hines, 55 F. Supp. 2d at 69.

d. Standards and Controls

The fourth Daubert factor considers the existence and maintenance of standards and controls. As noted above, Musheno bases his analysis on the FAVIAU checklist. He described this form as an "ever-changing document." See (Daubert Hr'g Tr. 93, May 6, 2005.) Rather than providing true standards for authentication, Musheno acknowledged that the form is an "internal checklist for the FBI" that has never been subjected to outside peer review.Id. at 99. The form simply lists characteristics that examiners thought might be important in detecting virtual images and manipulations. Id. at 92-93. Nonetheless, the existence of the checklist provides a standard methodology and some level of consistency among FBI examiners.

More disturbingly, no standards govern the conclusions that examiners draw from the completed form. There are no guidelines for the number or type of factors that should be present to conclude that an image is real. (Daubert Hr'g Tr. 35-36, May 11, 2005.) Indeed, Musheno testified that he would conclude that an image is not real if it "looks cartoonish." (Daubert Hr'g Tr. 22, May 6, 2005.) He went on to explain that he would reach the same conclusion "if there were something so gross, if the perspectives were so out of whack" and noted that his conclusions are "image specific."Id. While the checklist provides a modicum of standardization for Musheno's observations, no standardization at all governs his final determinations. This degree of precision is insufficient to qualify as expertise when a defendant's liberty is at stake.

Likewise, Musheno testified that there are no objective criteria for determining when degradation has made it impossible to reliably analyze an image. See (Daubert Hr'g Tr. 40, May 11, 2005). An analysis of the images at issue in this case reveals the significance of this shortcoming. Musheno acknowledged that all of the images were compressed digital images in JPEG format. See (Daubert Hr'g Tr. 102-03, May 6, 2005). He further agreed that compression destroys information from the original image. Id. at 103. If an image is manipulated before being transformed into a JPEG file, some of the artifacts of the manipulation could "possibly" be discarded. Id. Musheno testified that he could not tell how many times (or when) the images in question had been compressed; he could only say for certain that compression artifacts were present. Id. at 106, 114. He also conceded that all six images that he identified as "real" had added text, a form of manipulation that could result in the further loss of information from the original image. Id. at 105-06. As a result, Musheno may draw conclusions from incomplete information without knowing what details were missing or how or by whom the information had been removed.

It appears that the only objective standard to which Musheno adheres in making his final determinations is his practice of not declaring an image real on the basis of a single image.Id. at 23. There are serious problems with even that criterion. Musheno testified that he will not declare a single image real because he "know[s] it is possible to create imagery or to manipulate imagery that's undetectable. . . ." Id. He is, however, comfortable declaring an image real "if there are multiple images and they're of great detail and they're complex. . . ." Id. at 25. This is not because the analysis itself differs when multiple images are being considered.Id. Rather, Musheno stated that he is "aware of how incredibly difficult it is to either manipulate or generate one image alone. If you start to have multiples, then that difficulty increases." Id.

But as Frabizio rightly argues, this conclusion is based on the assumption that the computer artist started with a single image, rather than, for instance, alterable images of a single adult in multiple poses or from an entirely virtual image. This single and highly questionable criterion alone surely cannot satisfy theDaubert/Kumho Tire interest in the existence and maintenance of standards and controls. See Hines, 55 F. Supp. 2d at 69 (noting that one cannot "compare the opinion reached by [a handwriting analyst] with a standard protocol subject to validity testing, since there are no recognized standards"). See also Fed.R.Evid. 702 advisory committee's notes to 2000 Amendment ("The more subjective and controversial the expert's inquiry, the more likely the testimony should be excluded as unreliable.").

e. General Acceptance

Finally, there is no evidence that others in the field generally accept this technique. See Kumho Tire, 526 U.S. at 151 (citing general acceptance among its examples of appropriate applications of the Daubert standard to technical, experience-based fields); Hines, 55 F. Supp. 2d at 68-69 (applying the general acceptance factor to handwriting analysis). Musheno testified that his unit at the FBI has been conducting image authentication analysis since the mid-to-late 1990s. (Daubert Hr'g Tr. 15, May 5, 2005.) He later clarified, however, that his unit only began applying the technique in earnest within the last two to three years. (Daubert Hr'g Tr. 86, May 6, 2005.) It is clear, then, that Musheno's technique cannot claim a history of general acceptance. Rather, this technique is apparently the product of a group of FBI employees who endorse one another's work.

The criticism that the technique has only been considered by other FBI examiners is applicable to a wide range of technical fields. See, e.g., Hines, 55 F. Supp. 2d at 68 (observing that to the extent handwriting analysis had been "generally accepted," it was not by a "financially disinterested independent community, like an academic community") (quoting United States v. Starzecpyzel, 880 F. Supp. 1027, 1038 (S.D.N.Y. 1995). Many of these fields have withstood Daubert challenges. See, e.g., United States v. Crisp, 324 F.3d 261, 270 (4th Cir. 2003) (rejecting Daubert challenges to handwriting and fingerprint evidence). But fields such as handwriting and fingerprint analysis, though certainly not without their own problems, see Hines, 55 F. Supp. 2d at 68, can claim both more analytic rigor than image authentication and certainly a more venerable tradition of acceptance, id. at 69, 67. In Hines, for instance, I noted that expert testimony regarding handwriting analysis has been generally accepted for decades. Id. at 66-67. The same cannot be said of Musheno's image authentication technique. f. Helpfulness to the Jury

The First Circuit has held "`[t]he ultimate purpose of the Daubert inquiry is to determine whether the testimony of the expert would be helpful to the jury in resolving a fact in issue.'" Hochen v. Bobst Group, Inc., 290 F.3d 446, 452 (1st Cir. 2002) (quoting Cipollone v. Yale Indus. Prod., Inc., 202 F.3d 376, 380 (1st Cir. 2000)). In making this determination, the Court must consider, "given the proffered expert's background, whether the scientific, technical, or other specialized knowledge he offers will assist the trier better to understand a fact in issue." Gaydar v. Sociedad Instituto Gineco-Quirurgico y Planificacion Familiar, 345 F.3d 15, 24 (1st Cir. 2003) (internal quotation omitted). Although this inquiry is related to the question of the expert's reliability, the First Circuit has described assistance to the trier of fact as a "distinct" requirement of Federal Rule of Evidence 702. United States v. Shay, 57 F.3d at 132.

Clearly, Musheno's observational skills could be helpful to the jury, if the threshold concern about the possibility of wholly virtual images were addressed. His familiarity with Photoshop and the characteristics of images that have been manipulated would provide tools to assist jurors in their deliberations. Essentially, Musheno's experience with image manipulation would guide the jury. Unlike his observations, however, Musheno's conclusions assert a level of certainty unjustified by his methodology and experience. See, e.g., United States v. Green, 405 F. Supp. 2d 104 (D. Mass. 2005). In addition, excluding these conclusions, which are of questionable merit, would reduce the danger that the jury would cede its decision-making entirely to him.

On the current record, however, I cannot allow Musheno to testify even to his observations. As discussed in section II,supra, I find that the current state of technology prevents a lay juror, or any other visual observer, from assessing whether the images in this case depict real children, as opposed to computer-generated or digitally-manipulated images. Therefore, Musheno's observations — without the conclusions, which I must exclude as discussed above — are not useful to the jury.

g. Other Indicia of Reliability

In line with the Supreme Court's instruction that the Daubert factors should be applied flexibly to technical fields, it is appropriate to consider other indicia of reliability. Kumho Tire, 526 U.S. at 149-153, 156 (noting with approval that the district court had considered that there were "no countervailing factors operating in favor of admissibility . . . [that] outweigh[ed] those identified in Daubert"). The parties here have pointed to no additional indicia of reliability, but one additional element of Musheno's technique concerns the Court.

Musheno explained that, when evaluating images, he looks for signs of manipulation — that is, signs that someone had tampered with a picture of a real person. (Daubert Hr'g Tr. 90-91, May 6, 2005.) Again, this assumes that real photographs have been manipulated. As defendant suggested in cross-examination, however, a wholly virtual picture would not show the same signs of manipulation as would a real picture. Id. Thus, the possibility of creating images that are entirely computer-generated and which therefore do not demonstrate the inconsistencies Musheno said he would expect in a real photo that had been digitally doctored, creates substantial doubts about the reliability of Musheno's technique.IV. CONCLUSION

In a world of rapidly changing technology, where the availability and use of Photoshop and other, similar programs is widespread, substantial evidence suggests it may be possible to digitally create or manipulate photographs in a manner the naked eye cannot detect. The government has not shown otherwise. Under these circumstances, it is unreasonable to expect a lay jury to differentiate the real from the computer-generated. The government must therefore present an expert or other extrinsic evidence to prove that the images in question depict real children.

Whether the images in this case are real or virtual cannot be determined based on mere observation, however, even by a photographic expert. More specialized, computer-based knowledge is required to exclude the possibility that the pictures are wholly virtual. Furthermore, even if visual observation were sufficient, I would find that Musheno's qualifications and expertise do not justify the conclusions he proposes to make. I would allow him to testify to his observations, but not to his ultimate conclusions.

Here, however, the government has not established that the pictures at issue in this case are not wholly virtual or that visual observation can reliably detect manipulation, if any. I therefore GRANT defendant's Daubert motion, and pursuant to Federal Rule of Evidence 702, exclude Musheno's testimony in its entirety.

SO ORDERED.