Before we start, I'm going to force a little thought experiment on you. 
Imagine two politicians - we'll call them Purple and Lime.
Purple is given the power to determine the classification of narcotics under a system broadly like the British one. He's a completely reckless and often deluded libertarian. For ideological reasons, he decides to downgrade cannabis from class B to class C or whatever the equivalent is. In doing so, he does not consult any experts or scientists, and he convinces - through some process that doesn't involve justification of his beliefs - the Parliament or Senate or whichever bodies it needs to pass through. Perhaps he has dirt on all of them which he threatens to expose unless they do his bidding. As such, the change is made.
Lime is also given the same power to determine the classification of narcotics under a similar system. He decides to appoint a committee of scientific experts who are charged with providing him with the current scientific evidence which he can use in the process of making the decision. The committee returns that evidence and Lime decides that although the evidence presented is compelling, there is a strong ethical objection he and many other people feel to the change. He decides to reject recommendations of the committee.
It later turns out that the experts were broadly right in their conclusions.
If you are an advocate of "evidence-based policy", which of these two politicians do you think is acting more in accord with the idea of evidence-based policy? In Purple's case, we do actually end up having an evidence-based policy. The policy matches the recommendation of the evidence given certain values. Of course, this is by accident - we have something a bit like a Gettier case, only with the added confusion of some deontological stuff mixed in. In the case of Lime, I would tend to say that he has more of a claim to evidence-based policy since he has at least considered the evidence but then rejected it based on his values. Neither is ideal, of course. But in Lime's case, his decision is evidence-based - he's just chosen to ignore the evidence. The evidence is involved in the decision in a way it isn't with Purple.
The problem with evidence-based policy as an idea is that it ignores values and gives permission for technocracy. This, of course, never happens. You can't just say that you are going to avoid questions of values and then do so. Last night, I was at the Westminster Skeptics, where there were talks by Evan Harris MP and Professor David Nutt. It was certainly interesting to hear from Nutt - he has been sacked by Alan Johnson from the Advisory Council on the Misuse of Drugs, who make the scientific recommendations to the government regarding classification of narcotic substances under the Misuse of Drugs Act.
I went to the meeting with the same deep reservations I have about the idea of evidence-based policy, only to find that Nutt mostly avoided the more philosophical/political question, while Evan Harris had a surprisingly complex understanding of the idea. I think Harris is broadly on the right track.
I've raised political questions among my fellow skeptics before and I've heard plenty of responses like "oh, I don't really do politics - I think it should all be evidence-based, not based on any ideology or anything".
It is precisely at that point where I diverge. I don't believe that policy is a matter of fact. Some of it is, but most of it is value. There isn't a scientific test which tells you whether or not, say, smoking cannabis should be illegal or not, or for how long rapists ought to go to prison, or whether or not it should be legal to publish cartoons which some people find blasphemous.
Acknowledging that some questions aren't answered by evidence doesn't mean that we are freely jumping into the realm of irrationalism. We actually reduce the possibility for coming up with the sort of policies we want if we see policy in Manichean terms of either being evidence-based or superstition-based. Let's say we have a policy that is based on a shared moral belief about something - while we cannot necessarily find a gene or do a sociological study that shows that moral belief to be wrong. It may be based on a faulty description of the world. If I said that we must ban snorples because they are threatening our stock of wibble beans, I am just talking nonsense. If I said that the age of consent ought to be 16 because that's what the law says, I'm making a circular argument.
We should be insisting on logic from politicians. This is really the first baseline. We should hold politicians to a strong standard of logical consistency. You can't really be presuming P one week and not-P the next week. But I don't see it. Politicians must be making just as many logical blunders as anyone else in public life, but the media calling them out on their fallacious reasoning is few and far between. We see a media that obsesses over youthful indiscretions or what Americans call 'inside baseball' - the minutiae of Westminster life. We get politics all the way down to the micro-level of whether Brown meant a particular statement as a slight to Blair, but we don't get policy or analysis from the media. Most of all, we see public figures use deliberately inexpressive language that builds up straw men of their opponents positions, and these get duly reported by the media rather than explicitly called out as bullshit. That only develops further and further - allowing one group to build up straw men and dramatically knock them down becomes permission for the next.
All of these things can be criticised but they are not the same thing as evidence-based policy.
I'll give a perhaps slightly absurd reductio of evidence-based policy. We finally rid ourselves against the laws on rape when an evolutionary biologist proves to us that rape gives one a Darwinian advantage. If the only value that need go into policy making is what the evidence says, that's the admittedly absurd and slightly non-realistic end case. Now, nobody who is advocating evidence-based policy really thinks that way. But that's my very point. If you can think up counter-arguments to that suggestion, I'm betting that a fair few of them have nothing to do with science or evidence. They have to do with the rights of the victim, the moral duty of one person to another, the infringement on liberty, the duty to increase rather than decrease overall happiness, the duty to follow the law. I'm not really bothered what justification you use: I gotcha! Another way I'll get you: make a list of all the laws you think are barbaric, and then tell me why. I'm sure there's a deontic justification in there somewhere. Even the ardent moral relativist makes moral claims - they say 'no moral conclusion can be held firmly'. Why not? Because they say you aren't allowed to. Who says? The person who is making a firm moral conclusion despite saying that you can't.
David Nutt did it last night: he said on one of his slides that we should "take the moral argument out" of the discussion, then in the next slide but one has a quote from On Liberty by John Stuart Mill. Sorry Professor, that hasn't taken the moral argument out of the discussion. It's quite blatantly left it in the discussion.
The terminology of evidence-based policy makes it seem like 'evidence-based' is a binary predicate, a property that either exists or does not exist in a policy. It's not though. There is a rich interplay between value and fact. Either a policy is based on evidence or it isn't. I don't buy that. Different parts of a policy are justified by different values and by different sets of evidence. A law regarding the use of NHS funds for certain types of treatment is going to draw on some values like "it's generally better if people aren't ill" and "government funding is a desirable method of providing healthcare" as well as facts concerning finance and facts about clinical practice and the results of medical research. If your measure of this policy is whether or not it is 'evidence-based' or not, you are only really seeing one half of it - it's evidence-based and it's value-based. Ignoring the values doesn't make them go away. Far better to try and grapple with the values as well as the evidence, to try and see if the values are logically consistent, meaningful and don't rest on bad epistemic habits.
None of this means I don't agree with the fundamental concern of the evidence-based policy crowd: that the government often plays loose and fast with the facts, and with the testimony of scientific experts. But I think the underlying problem is both a disrespect for facts and some kind of reluctance to discuss the values upon which those facts are relevant. I think the solution is to demand both that politicians are scrupulous and honest when they are dealing with facts, but also scrupulous and honest when dealing with values too - they should openly declare them, and stick to them. The values that are guiding the policy should be clear, open and out on the table for everyone to see and, most importantly, question. Those with sensitive consciences need not apply for a job in politics. Truth - or consistency with the truth - would quite naturally be one of those values under discussion, and hopefully one of the strongest of the values.
One final example which I hope will seal my point: a lot of people have been talking about torture in the last few years what with Guantanamo, Abu Ghraib and so on. A key point many seem to bring up is that "torture doesn't work". Okay. That's not really the primary complaint though. This is one of the few things which almost everyone agrees to: torture is wrong. We've written it into laws, conventions, constitutions, ethical principles and even theologies. Torture is one of those things that sorts the real relativists from the undergraduate posers! The prime complaint against those who practice or authorise torture isn't that it doesn't work. If that was the only thing wrong with torture, then the torturer would have no moral significance: he'd just be someone who made a mistake. Robbery 'works': the criminal sometimes walks off with a large bundle of cash. But it's wrong.
I hope my thoughts on this aren't too rambling - it is 4am though. Do go and read Expert advice, an excellent post by Janet Radcliffe-Richards on the Practical Ethics blog.
