More thoughts from Chicago
This is a continuation of my thoughts on the American Academy of Forensic Sciences Meeting. A benefit of programs like is the opportunity to meet nationally recognized experts, and see how they approach things. Unfortunately, there is a pretty big disparity in the quality of forensic scientists; if you don't know how the good ones approach cases you don't know what you are missing. Unlike the experts you often encounter at trial, most of the presenters at the program didn't appear to be all that impressed with themselves.
One of the more interesting speakers the first day admitted that he could have – and should have – been attacked when he testified as an expert. He set out some things to look at to determine whether testimony meets the “crap test”. Basically they were areas you should look at, which included:
- Interpretation – what was the result of interpretation, and is it valid and supported
- data – what is the opinion based on
- report – how complete is it (if there is one), and does it explain the process used and any assumptions made
- variability – where evidence is tested, was there an adequate sample. This is particularly important in drug
- references – are any cited or relied on
- skepticism – this has to do more with peer review. Has someone else looked at the opinion and tried to find the holes that exist. Obviously in most cases there is no such thing.
Looking at those factors, it is a pretty good checklist to use when evaluating forensic evidence. One of the most overlooked factors – at least in my experience – is the assumptions used. To reach an opinion about anything, you have to assume certain things. If the assumptions are not valid, then the opinion cannot be relied. You don't know if the assumptions are valid unless you know what they are.
Another factor that is often overlooked is the data relied on. An opinion is only as good as the data it is based on. Many times, all the data is not set out, and it is our obligation to find it. You may have to look hard – and dig deep – to discover what data was used.
The final thing I took away from this presentation – as well as several others – was the idea of skepticism. As I set out yesterday, there is usually a monopoly of opinion. When someone else does look at a report – e.g. fingerprints – how rigorous is it. Do they really scrutinize the results, and try to prove the opposite? I think more often than not they merely look to determine if its close. If he State won't do this, then it becomes our obligation to put the evidence to the test.
Another presenter focused on something I learned a long time ago. If an expert asks you to "trust them", red flags should pop up immediately. A good expert should never have to ask anyone to trust them. If they can't explain their opinions and trust you to reach the right conclusion, then they probably don't know what they are doing
Another red flag is "based on my training and experience". Just because you have been doing something for along time doesn't mean you are good at it. Nor does it mean you cannot make mistakes. Any doubt should be put to rest by looking at the the Brandon Mayfield case - where the most experienced fingerprint examiners in the country all got it wrong.
I left Chicago with my head about to burst, If I learned anything it was that I don't anything. It also reinforced my opinion that we can do a lot more to challenge forensic evidence - especially it is critical to the State's case. We all have a long way to go to educate ourselves on the forensics involved in our cases. We have to do it though - our clients freedom depends on it.
