Why do experts get a free pass?

I've written several times about the use of forensics in criminal cases and the problems that result when you allow scientists to decide guilt or innocence. Most recently I noted the horrible decision from the Court of Criminals in Ex ParteRobbins that allows convictions to stand even when an expert admits they got it wrong.

The fact that the expert  in Robbins reviewed the case at  all  was amazing enough. Most of the time once a conviction is obtained everyone moves on to the next one. It is up to the defendant to initiate review, and since most defendants don't have lawyers at this point that means the chances of having the case re-reviewed are almost - zero. They  have to be lucky to enough to find a sympathetic lawyer, or have family who know what they are doing.

What  would happen if experts routinely reviewed their prior cases? Or had a duty to do so? Well apparently they do in most places - just not the United States.  I recently came across a blog post titled "Honesty in Forensic Science", written by Anna Sandiford. She had the following to say:

At the end of the day, lawyers are there to deal with the law; forensic scientists are there to deal with the science and it shouldn’t be left to defence lawyers (or prosecutors, come to that) to dig around to see if, on the off-chance, they can find anything amiss with the other side’s expert’s work.  One of the basic rules of examining witnesses is never to ask a question to which you don’t know the answer.

Many times I hear lawyers say how they hated science at school and how having to deal with forensic scientists makes them go cold at the thought – it has to be the duty of the expert to deal with the science and its meaning.  Of course, the reason that independent forensic scientists like me advocate for review of all science being presented in court is so that we can pick up problems like changes in findings that haven’t been notified to the court – not all scientists are transparent about what they do and there are also accidental errors and omissions – we are human after all.

Much to my surprise, she pointed to the guidelines for experts in England and New Zealand which place just such a duty on experts. Too bad we don't have that here.

While you might think this would create a tremendous burden on scientists I don't believe that would be the case. The situations where science changes to a significant degree are relatively few, and scientists know when that happens. They also know better than anyone what cases might be effected, so it makes sense to place the burden on them. And while it is a burden, is it really too much ask someone to go back and re-evaluate an opinion that put someone in prison, or got them convicted?

If experts don't do this, then its up to the courts, and maybe even the legislature. Unfortunately, we've already seen the court's are not the place to resolve the issues, and most representatives are far more concerned with getting re-elected than doing what is right. So perhaps that means the battleground is public opinion.

I'm not sure what the solution is, but the first step is in recognizing there is a problem.  Surely that shouldn't be that hard.

More thoughts from Chicago

 

This is a continuation of my thoughts on the American Academy of Forensic Sciences Meeting. A benefit of programs like is the opportunity to meet nationally recognized experts, and see how they approach things. Unfortunately, there is a pretty big disparity in the quality of forensic scientists; if you don't know how the good ones approach cases you don't know what you are missing. Unlike the experts you often encounter at trial, most of the presenters at the program didn't appear to be all that impressed with themselves.

One of the more interesting speakers the first day admitted that he could have – and should have – been attacked when he testified as an expert. He set out some things to look at to determine whether testimony meets the “crap test”. Basically they were areas you should look at, which included:

  • Interpretation – what was the result of interpretation, and is it valid and supported
  • data – what is the opinion based on
  • report – how complete is it (if there is one), and does it explain the process used and any assumptions made
  • variability – where evidence is tested, was there an adequate sample. This is particularly important in drug
  • references – are any cited or relied on
  • skepticism – this has to do more with peer review. Has someone else looked at the opinion and tried to find the holes that exist. Obviously in most cases there is no such thing.

Looking at those factors, it is a pretty good checklist to use when evaluating forensic evidence. One of the most overlooked  factors – at least in my experience – is the assumptions used. To reach an opinion about anything, you have to assume certain things. If the assumptions are not valid, then the opinion cannot be relied. You don't know if the assumptions are valid unless you know what they are.

Another factor that is often overlooked is the data relied on. An opinion is only as good as the data it is based on. Many times, all the data is not set out, and it is our obligation to find it. You may have to look hard – and dig deep – to discover what data was used.

The final thing I took away from this presentation – as well as several others – was the idea of skepticism. As I set out yesterday, there is usually a monopoly of opinion. When someone else does look at a report – e.g. fingerprints – how rigorous is it. Do they really scrutinize the results, and try to prove the opposite? I think more often than not they merely look to determine if its close. If he State won't do this, then it becomes our obligation to put the evidence to the test.

 Another presenter focused on something I learned a long time ago. If an expert asks you to "trust them", red flags should pop up immediately. A good expert should never have to ask anyone to trust them. If they can't explain their opinions and trust you to reach the right conclusion, then they probably don't know what they are doing

Another red flag is "based on my training and experience". Just because you have been doing something for along time doesn't mean you are good at it. Nor does it mean you cannot make mistakes. Any doubt should be put to rest by looking at the the Brandon Mayfield case - where the  most experienced fingerprint examiners in the country all got it wrong.

I left Chicago with my head about to burst, If I learned anything it was that I don't anything. It also reinforced my opinion that we can do a lot more to challenge forensic evidence - especially it is  critical to the State's case. We all have a long way to go to educate ourselves on the forensics involved in our cases. We have to do it though - our clients freedom depends on it.

What is scientific testimony?

The Star Telegram had an article addressing the debate over the use of forensic testimony in criminal cases. They had the audacity to point out the conclusion of the National Academy of Sciences report that ballistics is not science - it's an opinion. Of course, that drew responses from crime labs, and those who regularly testify on such matters. The point of the article was not to answer the question of whether it's science or not, but point out the issues are currently being debated. In Texas, these type of issues were supposed to be addressed by the Texas forensic Science Commission.

Lawyers are known (despised) for arguing over words - yet we don't challenge the use of the term "science". So what is the definition? Here's one:

The word science comes from the Latin "scientia," meaning knowledge.

How do we define science? According to Webster's New Collegiate Dictionary, the definition of science is "knowledge attained through study or practice," or "knowledge covering general truths of the operation of general laws, esp. as obtained and tested through scientific method [and] concerned with the physical world."

 As far as I'm concerned, a crticical part of science is utilization of the scientific method. What does that mean? While its been a long time since I was in school, it basically means formulating a hypothesis, and then testing to see if its true. Rigorous testing means attempting to disprove the hypothesis. If you can't disprove it, then it's probably scientific.

So how does this apply to fields such as ballistics, fingerprints and blood spatter? I don't think it does. The underlying assumptions for those disciplines  has never been established: i.e. there are no studies establishing that no two fingerprints are the same, or guns put the same marks on bullets to the exclusion of all other guns. Ultimately, all those fields are based on one thing - observation.

The problem with observation is that no matter what procedures you follow, the ultimate decision is subjective.That is the problem with those fields; if something is true, everyone that looks at it should reach the same conclusions. If you have two equally qualified experts reaching different conclusions, that's not scientific.

Another problem is determing who is an expert. Is it someone who looks at a lot of fingerprints or ballistics. Does that mean they are able to accurately look at something at make comparisons. Why don't we use artists - who have a discerning eye for detail? Also, if it's nothing more than making comparisons, why do you need an expert?

We need to start addressing this type of testimony for what it is - opinion. What's the answer? I don't know. Perhaps its the use of court experts, or perhaps its in limiting the use of such testimony, or providing limiting instructions. Something needs to be done, because there is no doubt that as long as you have competing opinions one of them is likely to be wrong - and it is justly likely to be the state's expert as the defense expert.

How long does it take to become an expert?

ne of the blogs I enjoy reading is Zen Habits. I've picked up some great tips organization, productivity and keeping things in perspective.  Sometimes there is also information that specifically apply to lawyers. recently he wrote about expertise, and how long it takes to achieve expert status - his answer was at least 6 years.

I'm sure that's not what new lawyers want to hear; they come out with a law degree, and consider themselves experts. Many aren't shy about telling you about their expertise; just google criminal lawyer or any variation thereof. There's no criteria for when you can declare yourself an expert - its up to each lawyer to decide themselves. Unfortunately, most of those who consider themselves experts aren't; and those who don't consider themselves experts really are.

I agree to a point with Zen Habits - you can't become an expert in anything less than 6 years. However, just because you've reached that mark doesn't mean you are instantly an expert. I wish that were the case - I've got 27+ years under my belt. That means I've been around, but doesn't make me an expert. I like to think I'm really good at certain things - but certainly not everything that relates to criminal law.

When I first started I had an older lawyer tell me it took at least 5 years to figure out what you are doing and get a practice started. I thought he was crazy, but he was absolutely right. I've seen it time and time again, and finally started giving the same advice.

Expertise requires more than knowledge. It also requires experience. More importantly, it involves perspective. Not only do you need to know how to do something, you need to know when to do it. Contrary to what most people think (especially defendants and new lawyers) practicing law requires more than knowledge. Good lawyers have intangibles that others don't possess. One of those intangibles is a passion for justice and a desire to ensure their clients are treated fairly; many times those desires conflict with economics. In other words, if your only interest is in making money, I don't think you are ever going to become an expert.

The internet is a great thing - but it can't make you an expert without experience and desire. you still need to put in the time. sorry - there is no short cut.