Wednesday, October 14, 2009

Harvey Milk and Phenomenology

I was listening to an interview with Chris “Mad Dog” Russo on the radio this morning, and I was struck by his citing the film Milk when asked about his view on same sex marriage. Mad Dog is a New York based sports talk show host, for those who don’t know. I’m not saying that all sports talk show hosts are conservative, but from my years of listening to WEEI in Boston, at least on that station, liberal points of view are rarely spoken, and often mocked—by most hosts and listeners alike. Not all, of course. But anyway… The host this morning asked Russo if he was for or against gay marriage and there was a long pause. He balked a bit. But in answering the question, he brought up Milk implying that after seeing this movie, he realized that this was an issue about real people, etc., and that he is for same sex marriage now. (This may not be the entirely nuanced version, but it hits the main points.)

Now, what does this have to do with this blog? Everything. What I take from this is a very clear example of how popular media change public opinion. Popular media change public opinion. Russo was not persuaded by logic here; he was not reading the academic journals on queer theory and persuaded intellectually… He saw a movie, a story, became familiar with the life of one person, and his opinion (and I’m assuming many others’) was changed, or at least began to crack. The initial mission of this blog was born out of a frustration, a helplessness, that all of the good done within the walls of the academy—specifically educational research—go unnoticed or even actively disregarded by the public. But public opinion is what drives policy, which drives assessment, which drives curricula, etc. The mission of this blog was to take issues that tend to have meaning and support only within the academy walls and to get them out. And hearing Russo talk on the radio this morning, I thought it was worth sharing that story.

This, to me, is the point where we need to examine the role of scientific inquiry and the uses of scientific paradigms. That’s why I slipped the word phenomenology into the post title. What I heard in Russo was a man convinced not by logic and reason, but by a glimpse of the lived experience of a gay individual. So what does this say about scientific methodology? What is the proper role of experimental science in a field that means very little unless it can affect public opinion? Here’s the quandary: we like to think that we are ultimately rational, that we are persuaded by reasoning—by logos, not ethos or pathos. But that’s just not the way we operate. Why should it be? I am not saying that experimental methodologies have no role in transforming public opinion—of course they do. But I think we need to shed the naiveté that says they are all that shapes public opinion.

Validity theory—the study of what makes scientific inquiry valid—now considers validation to be an argument, not just a factual number. I mention this in another post. What’s important about this notion that validity is an argument is that arguments only have strength in relation to a given audience. This is huge because the old views of validity treated validation as if it could be done in a vacuum, as if social values had not effect, but they do. What I’m getting at is the validity—the necessity—of scientific methods that expand beyond experiments and that reach their intended audiences. An argument that is valid for one community may be invalid for another. In order to really get this idea, we have to really suspend our disbelief about some social values we take to be universal. For example, the prototype of experimental science, the double blind controlled drug test. This is a form of argument, and it only works on communities that value that type of evidence. We tend to believe that after a series of experiments that show that a drug has intended effects and minimal unintended side effects—enough to convince the FDA—that the drug is worth using. The experiments were valid. But experiments, if you recall from the other post, cannot be valid or invalid; all they can do is provide evidence for a decision, and that decision is where validity lies. So if you were to take a community of people whose religion forbade any kind of drug and show them all the statistical evidence in the world, all the results of the controlled randomized experiments, etc., you would not convince them that taking the drug was their best decision. The experiments have no validity for this audience.

One assumption we have to unpack is something that we tend to believe so strongly, we consider it universal. We tend to believe that life should be extended as long as possible, almost regardless of quality. It’s kind of hard to debate that, as a society, we hold this value. So, holding this value, a drug that will maintain life is a valid piece of evidence for the decision to take it. But if we value a life of purity over longevity… then maybe this scientific methodology is invalid. Validity theory is a hell of a drug!

The point here is that we throw around the words scientific and unscientific is careless, even harmful ways. If you are trying to convince a population that condoms are good for preventing HIV, but you don’t understand the social values that determine what type of argument is persuasive and what rules are worth obeying or breaking, what lines of evidence are convincing—if you assume that all societies are convinced by what we tend to call scientific method, then you are in trouble. Narrative evidence—like a film that chips away at a person’s long standing belief that homosexuals are somehow deviant—this is in some cases much more persuasive than other types of evidence, such as experimental or statistical. So, am I calling Milk science? Of course not. But I am arguing that phenomenology—the method of investigating lived experience—is science, and it’s effective, even more effective than experimental methods depending on the audience. (Yes, science has an audience and must adapt its methods accordingly*.) I’m arguing that we privilege on paradigm of scientific methodology with no real reason for doing so, other than it’s just what our cultural values dictate. So much so that I assume some people reading this are thinking that science is just science, that it’s just right and everything else is just guesswork. But I think this is something we really need to fight about. I am arguing that we need to loosen this grip, especially when US reports call for more scientific evidence-based research on education. Because phenomenological research IS evidence based, and it IS highly scientific. It’s just not experimental or statistical. It provides insight rather than proof sometimes. But think about how powerful insight can be, and how in the face of tons and tons of “proof” we are able to hold on to our beliefs anyway.

Just some stuff to think about in terms of research and science, and how we need to get our research outside the limits of the walls of our tiny scientific communities if this research is to mean anything to anyone other than us.

9 comments:

  1. I feel like you are conflating science and public policy in a way I don’t always follow. The role of scientific inquiry is not to convince the greatest number of people. In the sciences, it matters what you can experimentally demonstrate, not whether you can persuade people of a particular point of view. The scientific process hones in on fundamental physical truths that don’t give a damn what we think. This is to be contrasted with issues of social justice, like your example of gay rights: the scientific method has no empirical answer as to whether or not gay people should be allowed to marry, because that’s not an empirical question, it’s a moral question. When you say that, “arguments only have strength in relation to a given audience,” this is simply not true about empirical arguments; it’s true of other kinds of arguments (i.e. normative claims.)

    Thus I can’t agree with you that a scientific methodology can be considered “invalid” because it fails to persuade a given audience: it is not the function or the intent of empirical methodology to “convince” -- that’s the function of normative arguments, public policy, and yes, popular media / propaganda. This is why the scientific paradigm is categorically different (or “privileged”, as you put it) – not because we place arbitrary cultural value on it, but because it has a correspondence with objective physical reality (“what is the speed of light?”) that doesn’t exist in normative issues (“should gay people marry?”).

    I understand that you are ultimately concerned about what the applications of the scientific paradigm are. (I think this is what you mean when you talk about “scientific methods that expand beyond experiments and that reach their intended audiences.”) I agree that we throw around words like “scientific” willy-nilly, and you can’t always assume that people will be rational interpreters of empirical data. But this fact says nothing about the scientific methods themselves.

    As a second observation, I also think you underestimate people’s capacity to be swayed by logical evidence. No, we certainly aren’t good at being rational actors – anecdotes and experiments abound that illustrate how terrible we are at interpreting, weighing, and acting on evidence. We have the capacity to cling tenaciously to beliefs even in the face of massive contradictory evidence. But that doesn’t mean evidence is irrelevant to us. We make evidence-based decisions every day about all sorts of mundane things; it’s in our nature. Moreover, it's a tendency we should encourage.

    There may be a place for anecdotal and “narrative evidence” (is that term a contradiction by the way?), but to make a case for its importance, I feel like you need a stronger argument than “People often aren’t good at paying attention to empirical evidence.” Phenomenological research is evidence-based, yes, but it’s not very good at telling you the way the world is – as such, it can be a good tool for convincing people, but it’s a bad tool for expressing the larger picture of the way things are. I would fear a world in which we relied only on anecdote, narrative, and appeal to instict to convince a public – not only is that a cynical and paternalistic view of humanity, it opens the door for horrendous abuse; the best propagandists will call the shots. (Have you ever read about Turkmenistan?) Insight can be powerful, sure, but one person’s insight is another’s irrational bias.

    ReplyDelete
  2. Okay, take two--since my computer just lost my post!

    Travis, I think our disagreement centers around the term 'paradigm'. I wonder about your opinion on Kuhn's original usage, because I think some of your claims go right against it, even the notion that I'm conflating science and public policy. I'm not sure the two are ever separate, and that one of Kuhn's major points was that it is only within the walls of a given paradigm that researchers think they are looking at the objective facts of the world in a value-free way; but that the truth is they are always influenced by social and political pressures, but we only tend to notice this from outside the walls of that paradigm.

    Some of the claims you make seem to come from too far inside the walls of a specific paradigm, and I think this is why Kuhn's work is so important: he showed us that we never really view the objective natural world outside of a system of value-laden, theory-laden lenses. In fact, I see the term 'paradigm' as a criticism more than anything because the effect is that it lets researchers go on unquestioningly paying attention to some, and ingoring other, phenomena--without even realizing that this is what they're doing. Kuhn pointed out that it is often the very phenomena ingored by one paradigm that may lead to revolutionalry discoveries in a latter paradigm. (My computer is totally fucking up, so please forgive typos)

    I think that today, especially after we have the advantage of being aware of the concept of paradigms, that we owe it to ourselves to NOT fall into the trap of ignoring phenomena that don't fit into the world we intended to view. And yes, I think this must include social phenomena. I think that since we are now aware that science never comes paradigm-free, we must commit to at least unpack the assumptions of the paradigms within which we operate. And I think the social sciences to a much better job of this than the natural sciences today.

    So I don't accept the idea that scientist are, or ever have been, looking at the objective facts of the natural world that don't give a damn about us. I just don't think that's what they/we see. And I think this is supported by much philosophy of science, especially Kuhn.

    This sounds like good new post material.

    What do you think?

    ReplyDelete
  3. I'm clearly behind on my reading, here (Kuhn, 'paradigm') but please bear with me and help me out as I try to sort through this.

    As deeply as it pains me, I must as a scientist agree with the general crux of what Travis says above - with a list of caveats...

    Josh:

    I will grant that when I speak of science as concerning only objective reality, I am speaking about science as a kind of ideal - one that may not ever be realized perfectly. But surely you'll grant that at least some of the pursuits in the natural sciences ('What is the precise speed of light?' is a fine example) can proceed with little risk of cultural interference. I have a hard time imagining race, religion, or politics coming into play much on that one.

    I find myself hungry for examples. Can you explain what you mean when you suggest that science is inseparable from public policy, and provide some examples? Are you getting at the basic practical issue that often scientific research is funded by government? Or are you getting at something (even) more fundamental?

    I will certainly grant these points:

    (1) One finds natural scientists who treat their work - and their belief in the scientific method generally - the way religious fundamentalists treat religion: there can be no other way; there is no other path to truth. A little ironic, given that fundamental features of a good scientist are healthy skepticism and a wide-open mind, but there it is. Science is only as good as its practitioners, and we're all human. (And to the degree that such folks are just trying to be good scientists and nothing more, they're doing just fine; it's just that by ignoring everything that is not objective, they're exposing the great weakness of science.)
    (2) Science is a very practical thing; it's about what we can observe, and its merit is by definition gauged by its ability to predict, and by the degree to which multiple parties can repeat the same experiments and observe similar results. As such, it has tremendous limitations: e.g., it's not great with one-time events, and it's no good at all for subjective phenomena.
    (3) Science is a very _young_ thing; even though we have made a huge number of incredible advances over the millennia (and anyone who enjoys the use of light bulbs or computers is likely to be a fan of the scientific method and its role in technological development), there is infinitely much more that we can't explain. We can't even sort out _turbulence_, and that is a phenomenon governed entirely by natural laws that we understand very well. It's no wonder that natural scientists sometimes have a hard time considering social sciences 'real' science; we don't yet understand fully the workings of even a single brain, so how can anyone purport to study giant collections of brain-brandishing animals and call it 'science'?
    (4) Scientists _do_ get caught in ruts that are often the result of the framework in which they're operating - and great breakthroughs do often come in the form of ideas that simply don't fit in that framework. (Is this the point from Kuhn? I'm not quite sure, but if so, I'm there with you.)

    Anyway - all that said... I do think that in its ideal form (if maybe never its practical form) science is independent from policy or cultural concerns. And I do think that a fundamental feature of anything I'd want to call science is validation of hypothesis by experiment - even if the experiments are 'mere' natural experiments, as in astronomy (http://en.wikipedia.org/wiki/Experiment). Do you have potential counter-examples that you want to suggest? -Pursuits that you believe are scientific but that involve no experiments of any kind?

    ReplyDelete
  4. Man, a lot of stuff here! Let's start with the ending. I guess by experiment, I mean the traditional definition in which you actually manipulate two groups of things and apply your theory to one but not the other in order to see if it "works." That's maybe more of a "true experiment," and in human sciences, these are rarely ethical, though examples exist. But when you say experiment of any kind, then I wonder how loose we get. My definition of science is that it must start with observation of phenomena, develop and revise theory based on the observations, then apply the theory back to the world/worldly phenomena. Now, if that's an okay definition, then the third part could surely be called experimentation. But is experimentation only one way of applying it? I'm not sure. If the point is that the theory MUST be applicable to the world, then yes, I will grant that without this, we don't have science. But if it has to be through the manipulation of groups to measure the effects of treatments, then I don't think all paradigms of science rest on experimentation.

    The fact that govt's fund research is no small part of how politics and science are meshed.

    Not too long ago, people literally could not see that the earth moved because it contradicted too deeply held religious beliefs.

    Aristotle looked at the exact same phenomena as modern scientists, but he didn't see molecules, much less electrons. My point is that, sure, he didn't have the technology--but that is a huge point because it suggests further that we are not simply "looking at the world" but more importantly, he didn't have current *theory*. It's NOT the technology that allows physcists to "see" atoms, etc., it's the theory which literally guides their eyes, which literally tells when what to watch for (and therefore what to ignore).

    I come from philosophy, where the idea that we simply and objectively observe the world as it is--the cartesian model--was shed in the late 1700s with Kant, and has never been restored. It just doesn't make sense if you really think about it. But within the grips of a given paradigm, the perception is that this is what we're doing.

    Then, and Kerry, you'll love this, there's the "Whitey's on the Moon"-ification of it all. Who is in a position to be measuring the speed of light and why? There are surely political and social issues. They clearly reflect what a society values and doesn't. (This might sound anti-science, but it's not; it's just some form of example.) When Gil Scott Heron sang "Whitey's on the Moon," he was talking about a society that put untold millions into marking the moon with an American flag, all while people in the inner-city rotted away from poverty, filth, bad health care--all kinds of social injustice that these resources could have been pointed toward IF that was what we valued.

    So, my man, here are a few stabs at examples of how the illusion is that we just look at the world the way it is, but that politics, values, social pressures, and inter-generational theories stand between us and said world.

    (Read Kuhn's The Structure of Scientific Revolutions! I think you'd really be into it.)

    ReplyDelete
  5. Also, Kerry, Travis, let's kick this up a notch and see how far I can take it...

    I think it's hard to argue that the reason people like Aristotle or Newton didn't see quarks and photons was because of a) technology but b) theory too. And I think b) is the real key. That's step one.

    But there's the thing: theory is deeply entrenched in values. For one thing, from a sociolinguistic perspective, I will argue that all theory must exist in language; and languages are not just words and syntax but complex systems of values and beliefs that reflect the word view of the cultures through which the languages exist. So there's a circle here, and it all involves values--not a pure, objective "view" of the actual phenomena of the world.

    I'm sure that takes it a little far, especially for Travis, but I think there's an important point somewhere in there...

    ReplyDelete
  6. Let me see if I can distill your points down and reiterate them, so I can decide whether I agree with them. Let me know if I'm misunderstanding anything!

    Your definition of an experiment as an application of a theory back to the world from which it arose is pretty close to how I'd define a 'natural experiment' (one based only on observation, with no control group). I'd be a little more precise and explicit, and require that the aim of the experiment must be to test a hypothesis and determine whether it holds. But with that, then yes, I'm fine including natural experiments in the mix when defining science. (Otherwise, as you've noted before, we couldn't classify pursuits like astronomy as science, but I think such pursuits really are science.)

    Now, you have a perfectly reasonable point that back in the day when religion and other facets of culture put Earth at the center of the universe, the idea of a modern astronomical framework as we know it was unthinkable. But I think this is not incompatible with my point that while science in the real world is of course subject to the shortcomings of its practitioners, that doesn't mean that the ideal of science (perfectly objective pursuit of knowledge, independent from all of those human foibles) requires any modification. And I would argue (I think pretty convincingly) that we've made great strides toward that ideal since the Age of Reason began. Granted, we may have gone a little too far, and you can feel free to bring in your Kant here. But if we accept both the tremendous strengths _and_ the serious limitations of 'ideal' science, I think it's hard to argue against the claim that it's a very powerful tool and at least conceivably independent (in an ideal world) from sociopolitical baggage.

    Regarding Aristotle, Kant, and Descartes: I'll just reiterate two of my previous points. I grant that scientists work within frameworks of theory and technology that both provide power and impose limitations - and that don't necessarily have anything to do with the 'real' world. But that's just the nature of the business. Science is a practical matter. It's about developing models of the world that provide predictive power. I agree with you that science may or may not ever provide any knowledge about the 'real' world (and that's why I would never want to be _only_ a scientist) but if science comes up with a theory on a given subject that applies to every event observed in the past and continues to hold during all future observations of relevant events, then the scientist has done his or her job, and it really doesn't _matter_ what's 'really' going on; the theory is good enough. (Disclaimer: this is a scientist's argument - see Heisenberg, esp; I myself still want to know what's 'really' going on, even if that is impossible, but I still grant the power of science, and the independence of science from such a desire for such knowledge.)

    Re the Whitey's on the Moon business, sure, there are many interesting points there, though that all gets us into a new conversation. Of course, there was the Cold War and the Space Race to consider, and beyond that I give at least some credence to the argument that it's worthwhile to inspire a generation of scientists and engineers by making that giant leap for mankind (give a man a fish, etc). (We actually just had a great meeting with the DDR&E, who oversees DARPA and who reports directly to the Sec of Def, and one of whose four mandates, I was happy to hear, is 'Develop world class science, technology, engineering, and mathematics capabilities for the DoD and the Nation.') But all that is getting into social science, which we all know is not real science. :)

    I'll def go check out Kuhn at some point...!

    ReplyDelete
  7. Oh - and to bring all this back to your original post:

    I think we're in agreement that a sportscaster watching Milk isn't really science, but of course that doesn't mean one couldn't study such an event scientifically. Granted, it's social science, but still. (Heh.) Take a thousand gay-rights opponents, show them Milk, and compare their views before and after. No control group, but it's a reasonable natural experiment.

    Indeed, this is not dissimilar to one of the thrusts of my primary project right now (sponsored by the Office of Naval Research). We're looking scientifically at the effectiveness of various features of modern video games when applied in training domains. E.g., what are the effects on the effectiveness of a training game if it a numerical score is added? Competition among peers? Emotionally engaging narrative framework? ('Yo, baby! You dig it the most!')

    I'm also very much reminded of Jed's Public Diplomacy project:

    http://ashinstitute.harvard.edu/pdc/

    One of its fundamental assertions is that entertainment media of all kinds have effects beyond just entertainment - in particular, in public diplomacy and public policy. (And that we ought to pay more attention to these significant side-effects. _24's_ repeated theme that torture is an effective way to stop Bad Guys is an obvious example...)

    I'm also reminded of a recurring theme that we hear from commanders coming back from Afghanistan: the operations that most matter there are information operations. And though we have many many wonderful fancy killing machines, and gadgets to find snipers, make big battle plans, etc., the technology we are using most for those information operations is? Radio. Invented the century before last.

    So I certainly don't want to disagree with your claim that narrative is a powerful and often underrated and otherwise misunderstood tool. I just want to remain clear about the definition of science. -And to be clear that the latter can certainly be used to study the former, without loss of its original definition or standards.

    ReplyDelete
  8. Oh, and! I'm straying further and further from the original topic here, I realize, but I must relate this story, to hammer home the point that I believe I'm as much of a proponent of narrative in education as you, Josh. I've been attending these Chaos Theory lectures at work, by Steven Strogatz. The guy is freaking phenomenal. He presents the material (this is mathematics we're talking about here) by tracing its original discovery historically, as a _compelling epic mystery_! You're on the edge of your seat, and you think you might as well be sitting around a campfire, and suddenly you're like: Wait, I _finally_ understand the motivation and the genius behind integral calculus, and also orbital diagrams, and state space traces, and iterated functions! WTFBBQ?! Criminy. If every math teach was as good as this guy, I'd be flying a freaking plasma drive to Jupiter for a family vacation about now... Yo.

    ReplyDelete
  9. I think we're subtly dipping into the learning/acquisition schism... At least by analogy. That'll make a good post!

    ReplyDelete