Evidence-Based Belief and Practice

July 31, 2012

Interacting with some medical professionals, recently, has made me think a bit more about evidence-based belief and practice. I am in favour of evidence-based belief and practice; my ‘theoretical’ perspective is a broadly empiricist one, with an admixture of (to my mind) relatively sophisticated pragmatism. But what does evidence-based belief and practice consist in?

This is a more complicated question than it may appear, and these remarks don’t aim to do more than touch on the relevant issues. But, trivially, for our belief and practice to be evidence-based is for our beliefs and practices to be oriented to, and ‘checkable by’, the way things are in the world. In evidence-based belief and practice, we grant authority to empirical evidence to legitimise or de-legitimise our beliefs and practices. And, further, we evaluate this evidence itself by sets of rationally and empirically justifiable criteria, to evaluate which evidence counts as good evidence and warrants such authority, and which does not.

This granting of authority to specific types of event or entity – ’empirical evidence’ – is a social practice. Authority – on the Brandomian pragmatist metatheoretical approach I endorse – is created and granted by sapient entities’ social practices. We grant a specific social status to specific kinds of non-human things (pieces of evidence), such that these non-human things can possess social authority within human discourse. Once authority has been granted in this way, it cannot – again, as a matter of social practice – necessarily be easily revoked; this is one reason why, on a Brandomian account, the authority of evidence can go against all human preferences or authority-decisions, even when authority has its source only in human action.

In the ‘analytic’ philosophical tradition – as, often elsewhere – there is a tendency to assimilate the evidence-based responsiveness of the typical sapient organism reacting to environmental stimuli (the phenomenon of experience, or perception) to the formalised truth-seeking investigatory practices of the modern sciences. Willard van Orman Quine puts the point as follows:

The scientist is indistinguishable from the common man in his sense of evidence, except that the scientist is more careful.

I disagree with this assimilation: I think that the belief-forming practices of scientific investigation are quite socially and historically specific, and should not be seen as the extension, or fuller realisation, of more mundane and broadly-engaged-in practices of everyday empirical observation. Science cannot be defended on those grounds; it must be defended in its social and historical idiosyncrasy.

I believe this defence is a worthy one; I am an advocate for scientific practice. But engaging in this metatheoretical defence of science involves steering between two, opposing, flawed accounts.

On the one hand, if we understand science as through-and-through a social practice like any other, there is a temptation to see this perspective as robbing science of its authority (rather than as explicating the nature of its authority); this approach can therefore often lead theorists into a relativism that regards our choice of the scientific approach as arbitrary or unjustified. In classical social theory, this perspective is perhaps best expressed by Max Weber’s movingly pessimistic reflections in Science as a Vocation, where Weber’s own commitment to the social scientific endeavour is presented as an ultimately irrational obedience to a demon “who holds the fibers of his very life.” In more recent social theory, a similar perspective is conveyed well by the Edinburgh strong programme’s conviction that the social-scientific analysis of scientific practice leads, inevitably and correctly, to relativism.

Relativism – whether it understands itself as anti-science, as a consequence of science, or both – is a common object of critique. The opposing flaw is also a serious one, however: this is the perspective that grounds science’s authority in an appeal to the way things are in the world, without seeing how this appeal must itself be understood as a social practice embedded in a complex system of social practices. For this approach, in Hegel – and Marx’s – phrase, “the process vanishes in the result”: the mechanism by which truth-claims are arrived at is forgotten, and truth-claims are wielded as if they are the source of science’s social authority, rather than the result of that authority (as is in fact the case). These approaches, then, are dogmatic – they understand themselves as (and, in most practical contexts, are) pro-science, but they have an inadequate understanding of what science is, as a historically-specific set of social practices. Advocates of this perspective may be able to do science, but they are not able to adequately justify their findings, without relying on a tacit set of social norms that their dogmatism overtly denies. Many of the pugnacious contemporary advocates of science, like Richard Dawkins and Daniel Dennett, belong in this category.

If, then, we are to be good – or, more to the point, metatheoretically enlightened – proponents of evidence-based belief and practice, we need to steer a course between these twin dangers of relativism and dogmatism. This can comfortably be done – in the posts here on Robert Brandom I’ve gone some way towards explaining the broad metatheoretical approach that, to my mind, best enables such a position (though, again, Brandom’s work is pitched at the level of everyday empirical experience, rather than scientific practice). But I am interested, now, in beginning to actually do evidence-based work. I’ve still got a lot to do in elaborating the metatheoretical perspective I endorse; but I also want to begin to leave that space behind. Enough with philosophy; enough, especially, with ‘Theory’ that regards itself, in a smug but profoundly confused way, as ‘post-empiricist’. I’ve spent enough of my life in that space already. The task of social science is to describe and analyse the social world, through the collection and interpretation of data; that’s the project I’m committed to; one of these days I’d like to get to work.

Advertisements

20 Responses to “Evidence-Based Belief and Practice”

  1. lecolonelchabert Says:

    Have you read any of Roald Hoanns essays about the practise of chemistry. He always stresses the creativity and inspiration and diversity of talsnts necessary. So really perhaps there us a flaw in supppsing a practise of science gbat cpuld be described with an individual as example. The actual practise of science may involve necessarily a certsin number of people who answer quines description and a few people doinv sometbing different in a diiferent posture and mindset. The relatipnz between scientists who actually cannot be treated as micricosms of science could be the revealing object missed by empiricisms and pragmastism in their historical. ..not necessary…relation to liberal individualism.

  2. lecolonelchabert Says:

    Roald Hoffmann that shld say

  3. lecolonelchabert Says:

    Sociology…note…is most reluay advcanced by the worst most speculative practises. These require and structure rebuttals. I am thinking ic a) the mismeasure of man…a critique of a pseudoscience anx b) Capital…the critque of the pseudoscience of political economy. We want to think of social science as capital slone…marx is the scientist, Mandeville the pseudoscientist. But perhaps this is the bad robinsonade and the reason the querelle of relativism never goes out of fashion

  4. duncan Says:

    No I’d never read Hoffmann – am just looking at some pieces on his website now – very interesting.

    W/r/t the individual – I definitely don’t believe that science can be understood just by analysing the practices of an individual. By my lights, science is an intrinsically collective enterprise – one is only doing science to the extent that one is in some way participating in that collective endeavour. And I agree, one thing this means is that different individuals can fulfil different functions, have different dispositions, orientations, approach their work in entirely different ways, etc. and that this diversity of individuals’ practices can be essential to the formation of the overall endeavour – participation in which is what makes any individual’s practice scientific in the first place.

    I agree about the difficulty the liberal-individualist tradition has capturing or understanding this, though. Quine, for example, is systematically unclear about whether his ‘web of belief’ is that of an individual or a collectivity (he’s unclear because at some level he knows it must be the latter, but his entire apparatus is phenomenological-individualist, so he doesn’t have the resources to properly acknowledge this, account for what such a collectivity might be, or analyse it effectively. That’s quite apart (or maybe not…) from the fact that belief & experience, not practice, are his core categories).

    I also realise that ‘actually existing’ social science is in many respects a calamity – in that sense there’s a large amount of idealisation in my characterisations of the scientific enterprise. The question for me is what kind of idealisation this is though; how problematic. My view is that we can wield norms that are tacit within but central to the (social-)scientific endeavour to criticise ideological error (/deception) within the relevant disciplines. One therefore doesn’t need to reject (social) science in general, in order to criticise any given actually-existing social science (even the great majority of actually existing social science) – one can deploy social science against itself, wielding the best elements of the tradition against the worst, or indeed against the most common.

    I’d contrast this to an alternative view that sees social science as intrinsically and necessarily ideological in its very constitution, its fundamental presuppositions, etc. From this alternative perspective, social science doesn’t even contain the resources that could be leveraged into an adequate critique of its most ideological components; it’s rotten to the core, as it were. I would at one time have found this view, if not totally compelling, at least pretty persuasive. But I now think that it concedes far too much – it relinquishes the field of scientific investigation to ideological error, and there’s no reason to do that. That’s why I’d now be happy – even proud – to identify as a social scientist, when a few years back I’d have been profoundly wary of the label.

    I haven’t read that Gould book – I keep meaning to…

  5. lecolonelchabert Says:

    I also went through a phase of total yalederridean rejection of socsci. In uni. Before i became a media consultant and lrarned in practise how sophisticated and powerful are our methods of analysis of complex social data. Not to say ir isnt ideological dyed in the wool. Still with a critical marxist perspective theres so much value jn tgis lractise one has to assume the attacks of extreme socisl constructivism of the 80s was a reaganite ideological assat on radical scholarship. ..Twisting the antiimperialism behind canon revisionAfricana and womens studies,critical legal studies and critical race theory to an sbsurd nietzschianism geared to facitate the resurgence of europsupremacism we see now from badiou and niall ferguson and ditchkins. In the us some of the better writing on methodology of socsci is in anthro rather than the more politicised discipline of sociology. But perhaps for me the best statements of method are from historians like Carlo Ginzburg.

  6. lecolonelchabert Says:

    http://www.amazon.com/gp/aw/d/0874519330

    I got a lot from this and also his critique of Agambens obnoxious book about Auschwitz Levi Musulmanner Gorgon Grey Zone Heidegger the death camp as great work of art.

  7. lecolonelchabert Says:

    And intro to this http://books.google.gr/books/about/The_Judge_and_the_Historian.html?hl=pt-BR&id=_KTocO2QHpsC&redir_esc=y

    Wonderfl explanations from Ginzburg of the historical relations of medical practice,rhetoric,legal argument and the development of historiography

  8. ktismatics Says:

    I too am in favor of the systematic codification and improvement of work in order to achieve better outcomes. It’s clear though that the effort to establish evidence-based best practices is largely an attempt to transform professional practices — the result of training and experience and social norms — into mechanical procedures. Knowledge engineers make explicit the tacit decision-action criteria embodied by expert practitioners, while meta-analysis of empirical evidence seeks to improve on actual practice. Through the combination of these two methods it becomes possible to train either paraprofessionals or machines in performing expert work practices at high levels of competence. Like all automation, the results could potentially benefit the workers through shorter workdays, upgraded jobs at more pay for paraprofessionals, more time for experts to invent/discover better practices and so on, while at the same time reducing the price of products/services that rely on expert performance. Of course it doesn’t often work out that way.

    On the other hand, an evidence-based social science would be able to document empirically the extent to which specific automated work processes benefit/harm workers, managers, investors, consumers, and society. That sort of systematic study would go a long way toward demystifying the bullshit being slung by corporations and politicians about why the private sector is generating record profits while jobs are going away and paychecks are shrinking.

  9. duncan Says:

    a phase of total yalederridean rejection of socsci – yes exactly. And to be fair I should say that I learned a lot from Derrida & Derrideans – close reading skills, mostly – but also that the critique of ‘presence’ applies a non-negligible proportion of texts (and a high proportion of philosophical texts). But of course (I now see…) it all overreaches severely when it treats historical, empirical, sociological analysis as somehow lacking because it has not yet passed through or taken the measure of the deconstructive matrix. I was looking again, apropos this discussion, at the interviews in ‘Points…’ which in turn reminded me of Bourdieu’s line – “when I hear people say that Heidegger alone makes it possible for us to think the Holocaust – but perhaps I am insufficiently postmodern – I think I must be dreaming…”

    Ginzburg is yet another figure who is vaguely on the radar, but I haven’t read. If I ever do my history of capitalism project (as I hope to!) then presumably I’ll read him properly when looking at the early moderns… But maybe I should look at his stuff before that – thanks for the links…

    ktismatics –

    I wouldn’t myself draw too close a connection between evidence-based-practice, deskilling, and mechanisation. In my (anecdotal) experience of the medical space, for example, there’s a pretty clean correlation between skill level of practitioner and the quality of evidence underlying advice or interventions. Hospitals seem to me (again – anecdotally) to have a core of highly trained, highly skilled practitioners whose actions have a pretty strong evidence-based warrant, and then a delta of less skilled practitioners whose opinions seem to a fair extent untethered from a research-base. This is sort of what you’d expect, since that it takes more skill (and more knowledge) to make informed decisions than to make uninformed decisions – but for that very reason I wouldn’t draw too close a connection between evidence-based practice and deskilling. Though I agree there are connections to be drawn.

    Re: automation – I agree that these questions are of interest; my remarks toward the end of this old post may have some relevance here.

  10. ktismatics Says:

    There may well be national differences in the organization of healthcare delivery. My remarks were predicated on my own fairly extensive work experiences in evidence-based medicine in the US. I wouldn’t say that deskilling is the main issue. Physicians offload to nurse practitioners and technicians tasks that the doctors formerly performed themselves, thus upgrading the professional competence of these auxiliary health professionals. These offloaded tasks are performed often enough to justify precise specification of task sequences and empirical evaluation of their cost-effectiveness. However, these lower-status health workers don’t get pay raises commensurate with their increased skills and responsibilities, nor is the cost savings passed on to the payer. Instead the doctors, hospitals, and investors reap the profits from high-priced work performed by these lower-paid underlings. Of course this shift of high-level work practices to low-paid workers isn’t limited to healthcare.

  11. ktismatics Says:

    I don’t mean to be overly negative though, Duncan. As I said, I concur that establishing evidence-based standards for clinical practice is a worthwhile endeavor. It’s also kind of fun and satisfying. There was a time when I was better than most hematologists at calculating the proper dosage of clotting factor for hemophiliacs. And I also agree that physicians are reluctant to hand off too much of the work they deem essential to preserving their elite professional status and (at least in the US) their elite pay scale.

  12. duncan Says:

    Well I’ve never studied healthcare, so my remarks here shouldn’t be given any real weight – but yes, I see what you’re saying – and my sense is that the same trend is visible in the systems I’m slightly more familiar with (minus, to some extent, the for-profit dimension). Again, though, I’m uncertain how much I’d see this as principally connected to or driven by evidence-based (or not) approaches. I can certainly see that health outcomes might be improved in any given instance if routines that are demonstrably beneficial to patients can be instituted as a matter of policy, rather than relying on the specific judgements of artisanal highly skilled somewhat more autonomous health-care experts – and I can see how doing so would enable, and so perhaps naturally be associated with, the transferral of many tasks from those more artisanal experts to a broader set of auxillary health professionals whose competence would thereby be upgraded. I can also see how the evidence might point in the other direction, towards a decline in health outcomes associated with such ‘offloading’, in any given instance.

    I would guess, though, that the principal driver of such organisational transformations is often not, or not exclusively, the evidence itself but other factors: cost savings, a political preference for certain institutional forms over others, etc. I’m sure the extent to which this is the case differs in different times and places.

    What organisational and institutional forms provide the best health outcomes – as a (partly) separate question from what diagnostic criteria, medical interventions, etc., are regarded as best practice – is of course a social-scientific, not exclusively a medical question. It’s a question of great interest and importance. But it isn’t an area where I have any knowledge.

    (Btw, I wonder whether you have a narrower set of associations to the term ‘evidence-based practice’ than I do? In the sense I’m using it in the post, I see “evidence-based practice” as covering the whole gamut of medical practices grounded in the research literature – not as associated with any particular organisational trend; your remarks make me wonder whether in professional practice around medicine the term may have a narrower resonance, though.)

  13. ktismatics Says:

    We’re in agreement that clinical practices are embedded in broader social practices, which include the organization and ownership of labor. From the standpoint of management, empirical evidence includes not just health outcomes but also financial outcomes. Certainly corporate medical management in the US has demonstrated its ability to measure financial outcomes and to standardize specific work practices that demonstrably improve those outcomes. Health outcomes? Not so much. CEOs of hospitals and physician groups and pharmas are paid their bonuses based on financial outcomes, not health outcomes.

    But managers will certainly take health outcomes into consideration in maximizing financial outcomes. E.g., pharmaceutical ads and drug reps will tout marginally better health outcomes for some new drug that sells at 5 times the price of a generic. And hospitals will want to make sure that the less-expensive auxiliary health professionals taking on greater responsibilities aren’t killing people at any higher rate than the more expensive doctors who traditionally performed those same tasks.

  14. duncan Says:

    The US system is fucked up, there’s no question.

  15. ktismatics Says:

    True, but I suspect that fucked-upedness is a universal hazard. Have you had your wisdom teeth removed? Were the teeth infected/painful, or were they removed preventively? If you haven’t had them removed, does your dentist hound you to have it done every time you go in for a checkup? Is there an evidence-based controversy in Australia about costs, health risks, and health benefits of prophylactic wisdom teeth extraction? Here’s a 2007 evidence-based analysis of best practice.

    Regarding the offloading of high-priced work to paraprofessionals and machines, I’m an outsider to higher education but clearly the situation there is similar to that in healthcare, and my sense from the blogs is that it’s endemic at least throughout the Anglophone academic system. Grad students, adjuncts, and computers take on increased teaching loads: the financial outcomes are clear, but who pockets the cost reduction? And what’s the impact on educational outcomes?

  16. duncan Says:

    I don’t know what the typical Australian policy is on prophylactic wisdom teeth extraction. I think the NHS in the UK advises against it.

    Sorry – I don’t want to be rude here, but I’m not sure what you’re after in this interaction? I may be totally misreading your tone – in which case I apologise – but it seems like you’re angry about something connected to the topics we’ve been discussing, and I feel like the post’s advocacy of evidence-based practice and belief – and perhaps the passing reference to medical professionals in the first line? – has somehow placed me in the same space as the thing you’re angry at. That’s ok – but if you want to object to something I’ve said or that you take me to represent, you need to be a bit clearer about what it is.

    On this – “I suspect that fucked-upedness is a universal hazard” – well yes it’s a universal hazard, but of course not all hazards are realised. As you know the US healthcare system is notoriously poorly set up for health outcomes, relative to the systems of many other comparably wealthy nations: the country spends much more money on health care than comparable nations, with worse outcomes in many important areas. If your point is that every health system has some flaws, or that evidence-based practice can get it wrong too, then yes. But your affect (if I’m hearing it right?) seems disproportionate to those points?

    Again, sorry if I’m missing your tone here – but I don’t understand the points you’re trying to make in your comments.

  17. duncan Says:

    Reading back, I still think what’s going on here is that I’m using the phrase “evidence-based practice” in a broader sense than it’s used in the professional medical space, where I believe it must have specific associations that I’m not aiming for. It may have been a mistake on my part to use the term – I’ll try to think of an alternative form of words to make the point I’m trying to make in this post.

    To me, then, our point of disagreement is still this:

    It’s clear though that the effort to establish evidence-based best practices is largely an attempt to transform professional practices — the result of training and experience and social norms — into mechanical procedures.

    I disagree with this; training and social norms can also be informed by evidence, so the evidence-based / non-evidence-based distinction is simply orthogonal to the training, etc. / mechanisation distinction that you’re drawing. I think one of the reasons this conversation seems like a bit of a failure (at least to me! :-P) is that you’re interested in the latter distinction, and I’m still trying to talk about the former.

    Sound plausible?

  18. ktismatics Says:

    I got caught up in the medical lead-in to your post, which isn’t really the main point of the post as a whole. Of course medical practice should be based on empirical evidence linking process to outcome, even in the fucked-up US system. But what does it mean to be an evidence-based practitioner of social science? You say this:

    “The task of social science is to describe and analyse the social world, through the collection and interpretation of data; that’s the project I’m committed to”

    So sure, I’m all for that too. I thought that you were talking about formulating and disseminating socio-economic interventions as “best practices.” E.g., the IMF issues loans to national governments contingent on their privatizing utilities as a purported best practice for achieving economic growth. Is there empirical evidence supporting this alleged best practice? Or on what basis is it “best practice” in academic administration to shift teaching from professors to internet-based technologies? And so on. Okay, I’ll go do something else now.

  19. duncan Says:

    Glad we see eye to eye again!

    I realise it may seem sorta trivial to assert that social science should collect and interpret data, and make its claims based on the best available analysis of the evidence. It seemed worth saying here, though, because a) there are plenty of critiques of ’empiricism’ (or ‘positivism’) out there that don’t really accept this, at least for all kinds of evidence; and b) I think the Brandomian meta-theoretical apparatus I’ve been working on here for the last couple years can give a more robust and nuanced account of what ‘evidence-based x‘ consists in than is typically offered by many advocates of evidence-based belief / practice. So that’s what the post was ‘for’, at least from my pov.

    Anyway, glad we no longer disagree…

  20. duncan Says:

    Ok, so, don’t know if ktismatics is subscribed to this thread or not, but to return to his last comment –

    I thought that you were talking about formulating and disseminating socio-economic interventions as “best practices.” E.g., the IMF issues loans to national governments contingent on their privatizing utilities as a purported best practice for achieving economic growth.

    As you say, I intended ‘evidence-based belief and practice’ to apply more broadly than this – but of course one of the implications of having evidence-based beliefs is that you apply those beliefs in practice; and one of the areas in which such practices could be implemented is socio-economic interventions. So I’m not of course opposed to the idea of evidence-based socio-economic interventions – indeed, I’m in favour, and I think it would be good if public policy in general were more evidence-based. So in that sense, yes, that is what I’m talking about.

    Now something like the IMF’s neoliberalising interventions are, as you say, super-problematic examples of purportedly evidence-based practice in the socio-economic sphere. So I take it your criticism here is: how can the evidence-based practice I’m advocating for be differentiated, in practice, from such Bad Examples of purportedly evidence-based practice? Can they?

    I think there are two broad problems with programs like the IMF’s. First, political legitimacy (or its absence). I think public policy decision making should ultimately, and ideally, be grounded in democratic processes. (This doesn’t mean that political activity that isn’t participating in our democratic institutions or processes is for this reason illegitimate.) So I wouldn’t advocate, for example, a purely technocratic administrative and decision-making public policy system (or an authoritarian one). Evidence-based policy can only take us so far; there is also the question of political goals, and these should be decided upon through a decision-making process that includes as much of the general social collectivity as possible. And there is never a firm line between goals and implementation, either – the latter can and should also be open to contestation through democratic channels. There are, obviously, a heap of issues here around representation, delegation and expertise – and I should probably pay fuller attention to those questions at some point – but that’s the first direction from which something like the IMF reform programs can be criticised.

    Second, there’s the fact that the IMF programs weren’t really evidence-based at all; they were ideology-based. These policies were advocated by the consensus opinion of the relevant social-scientific discipline; but that discipline was wrong. I don’t think this is a problem with evidence-based decision-making, though – I think it’s a problem with our current mechanisms for making public-policy-oriented evidence-based economic decision-making. I suppose one of the projects I’d like to contribute to in the intellectual work I workshop here is the collective attempt to establish better mechanisms for socio-economic decision-making. So in that sense – yes, definitely, formulating and disseminating socio-economic interventions as ‘best practices’ is very much on the agenda here, though at a somewhat ‘downstream’ stage from where I am now.

    The two criticisms above are connected, of course: the fact that the institutions principally served by the discipline of economics do not represent, in their decision making, the hashed-out interests of the general social collectivity, but typically represent, instead, the hashed-out interests of a specific subset of that collectivity, is a large part of the explanation for why the opinion-forming mechanisms of economics are so badly skewed. That gets us into classically Marxist territory, of course. But at the moment I’m more interested in the processes of evidence-based research itself. Time enough to discuss that other stuff later (I hope).


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: