Goldacre, Gove and evidence

Something about Ben Goldacre’s recent report on Building Evidence into Education has left me feeling concerned. This feeling is not just caused by the “I love teachers” and “I wrote this lovely paper about lovely teachers” tweets, or the sense of medical research approaches and the “gold standard” RCT [edit: corrected from RTC, when it is, of course, Randomised Controlled Trial.]  coming to the rescue of the floundering field of education research, or the suggestion that there is no quantitative research expertise within existing academic education departments. It wasn’t even the fact that his report fails to mention things like Ofsted, teaching unions, teacher training, curriculums or do more than nod to the huge changes to the profession that would be required.

The thing that really worried me, I suppose, was that this report was commissioned by and launched while sharing a stage with Michael Gove. This is the same Gove who has consistently ignored evidence and expert advice in education policy on a whole host of issues, including, of course, the writing of a ludicrous history curriculum. As David Cannadine, who led a research project on the history of history teaching, wrote in the TLS a few days ago, Gove has in this case taken a course that is “the exact opposite of what has been repeatedly recommended by people who know what they are talking about”.

Is it simply that Gove has a greater respect for the evidence revealed by RCTs than he has for that produced through historical and sociological research? I am not wholly convinced. Does it seem likely that this government will start supporting state schools sufficiently to allow them time and resources to engage in research trials, not to mention the kind of pay scales that might be commensurate with a profession that undertakes such work and is listened to? Again, not convinced.

The report was launched at the Bethnal Green Academy, and it is interesting to put this report alongside Gove’s commitment to academies and free schools. It is surely schools of this sort, along with private and independent, rather than state primaries and secondaries – tied to government-defined curricula and competing  on the most unlevel of playing fields – that would be in the best position to attract and support the kind of teachers who could undertake research. This would only exacerbate the divergence in provision, with teachers in different kinds of schools working in increasingly different circumstances. In addition, it seems likely that evidence more likely be produced from particular kinds of schools, lacking any real randomness in the trials.

If the model is medicine, these schools are, like general practice surgeries, businesses. They too will be testing products and techniques offered by private companies, which, like pharmaceuticals, are in the business of finding ever-more conditions that require intervention. Meanwhile, Goldacre’s urging of teachers to take the initiative and ape they betters is as get-on-your-bike, pull-your-socks-up and self-help as any Thatcherite would like.

I’ll admit that this is not a topic I know enough about, and the foregoing may be complete twaddle. If so, I apologise, but would be interested to hear other views. I know that the schools I went to (long, long ago) and the school my son goes to today would not be capable of adding research – quantitative or qualitative – to what they already, just about, manage to deliver. I also know that much of education and education research is already a commercial business. Goldacre’s report won’t shift policy, but it is sobering to think about where it might sit in Gove’s world view.

————

Postscript [18/3/13 20:58]: Ben Goldacre has today published a piece in the Guardian on his report, which, despite a tweet that stated “People keep asking what I think about [Gove] and evidence. Here:”, did absolutely nothing to suggest he has thought deeply about the politics, about Gove or, indeed, about the school-based, education-focused context of his topic.

Since publishing this post, I’ve had some interesting discussions on twitter and been pointed to some useful posts and articles. Here is some further reading:

Behavioural Insights Team, published in collaboration with Ben Goldacre and David Torgerson, Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials – a policy paper from June 2012 that is fairly similar to the education piece, arguing that RCTs should be used widely in public policy, debunking some ‘myths’ about them then explaining how they should work in an ideal world.

Tom Bennett, ‘Thinking in the right direction; just don’t put all your faith in RCTs. Ben Goldacre’s vision for evidence based learning‘ – a post from an education blog that highlights the problems and impossibilities of the Goldacre vision. His description of Bethnal Green Academy – “Nice looking school; it’s got BSF written all over every plane and pane. The livery outside the school shouted every second sentence of the latest Oftsed report.” – was one of the things that got me thinking about the positioning of Goldacre’s report for Gove, although he also felt that Gove’s very presence was a positive sign for research and evidence in education.

Paul Cotterill, ‘The dangerous Dr Goldacre: Cohranian hero or just peddling garbage‘ – a post that points out the politics always and inevitably (and rightly) inherent in education and social policy, and that “a worthy aspiration can all too easily be hijacked by … governments (not just the current one) very keen on the idea of using research evidence to impose their own views of what success in education looks like, and less, but much less keen on the development of the kind of ambitious, democratically oriented research governance infrastructure that he advocates”. It also quotes from the following:

Will Davies, ‘The problem of evidence centres‘ – a post looking at the recently-announced evidence centres and the “somewhat naively optimistic view” that RCTs are a universal solution. But what are, and how do we define, ‘good’ and ‘bad’ outcomes in the many facets of educational experience?

Warren Pearce and Sujatha Raman, ‘Evidence: a means of making expertise public? The RCT movement in public policy‘ – outline of paper forthcoming at the  International Interpretive Policy Analysis Conference that will be asking “How can RCTs deal with the problem of expertise if experts are still needed to produce, interpret and apply evidence in particular circumstances? What sorts of institutions of epistemic governance might berequired to ‘open up’ RCTs vis-à-vis other forms of evidence for policymaking? To what extent have classical concerns about the ethical challenges posed by RCTs in social context been addressed in therecent literature?”

Dave O’Brien, ‘Drowning the deadweight in the rhetoric of economism: what sports policy, free swimming, and EMA tell us about public services after the crash‘ (Jan 2012 £) – an article that, exploring the Coalition government’s plans and decision-making processes in public service provision, notes “a specific rhetorical use of economic evidence to present controversial decisions as technical exercises”.

Advertisements

18 thoughts on “Goldacre, Gove and evidence

  1. All that Goldacre is saying is that it is desirable to know when an intervention/policy works and when it doesn’t. The principles of how to find out are much the same for social policy as they are for medical treatments. What could possibly be wrong with that?

      • :Let’s say Gove is delusional, I mean, just for argument’s sake: He actually believes what he says. (And btw Tom Bennett is a BIG fan.) So he’s standing there dreaming, “Finally evidence to support my belief that the 50s was the Golden Age.”

      • It’s possible he thinks that – this would be the outcome that he’s aiming for and the ‘good’ result of the trials implemented. I’m utterly depressed by that thought.

  2. Well, I’ll admit that I did find it a bit hard to see exactly what point you were making. Heaven only knows what Gove sees in it. Ben says that Gove asked him to write the paper. Is that really a Damascene conversion? If Gove were really interested in evidence he would have to address a large part of his ideology.

    • Are you saying that evidence should trump ideology in social policy? I thought the idea was for politicians to set ideological parameters for carrying out RCTs?

      • Unless I’m missing something this, the setting of ideological parameters, is the only serious challenge to RCTs. The only way to preserve your ideology is to restrict the evidence the RCT can produce.

      • A pretty serious challenge, though, even if it was the only one. However there are others – e.g. possible size of studies, type of questions that can actually be dealt with this way, training of teachers as researchers, how to create a ‘placebo’ for control, how to double-blind the teachers who must surely know what intervention they are giving (since they are an inextricable part of it), whether frequent testing of interventions and novelties is helpful in a child’s education at all —- and more

  3. Trouble is that everything changes very quickly in education so it is difficult to see if one intervention or another has had a positive effect. For example, if a pupil attends a STEM club in Yr 7 does that mean that they get better results at GCSE? Does it mean that they are more likely to do a STEM degree? If they do a STEM degree and say that attending the club made them want to do a STEM degree can we take their word for it? What else has had to happen to get them there?

    • Thanks for this. It also strikes me as potentially problematic if children are regularly and randomly assigned to test groups, especially given the rate of change in teachers, topics, individual circumstances, ability etc. Consistency in approach may well be more important than anything else. On-going education is not the same as having a medical condition, which is, one hopes, a temporary affair and, in any case, something we want to end (clear outcome).

  4. It’s surely just another part of Gove’s desire to portray anyone in the education ‘industry’ as incompetent. He probably doesn’t have any more actual respect for medical evidence etc than educational – since he’s a Tory politicians and former journo i.e will only listen to the evidence if its fits his view of the world.

    • “Michael Gove has expressed huge enthusiasm for these ideas. But many teachers say he ignores evidence, and challenge him to subject his own edicts to rigorous testing. Fair enough: I think good-quality evidence matters, whether it’s in education, medicine, or public policy, and I will stand on the barricades with anyone who wants to make that happen.

      Any health minister who tried to force doctors to use a treatment that doesn’t work would be laughed out of town. If teachers really want to force all education ministers to follow the evidence, there is one thing they can do today: work towards making evidence-based practice the everyday, unarguable routine – starting now.”

    • Yes, I’ve read the report, which largely ignores the issues of real-world implementation that I suspect would not be trivial. And your “serious challenge” still remains.

      • to clarify again: It is trivial to say there are significant challenges to design of RCTs. The challenges themselves.may not be.

  5. There are plenty of challenges in designing a good RCT that will be applicable to other schools, beyond the enormous logistical and ideological issues. How can you make sure that there are not significant interaction effects? How do we know what is happening outside of the classroom and in informal learning? Would it not be better to use data mining techniques?

  6. @Liz
    No it would not be better to use data mining techniques because they could not tell you about causality. Until the social sciences understand that establishing causality is the main obstacle to finding out whether one method is better than another, they can’t be taken seriously as science. And, more importantly, children (and teachers) will continue to have methods foisted on them which may well be ineffective.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s