Validity and Originality

Is it possible for a completely original qualitative method or measure to be valid? I have been toying with this question because I have been rethinking validity in general, having bungled the matter somewhat in my peer review, and because I have been wondering about the legitimation of methods and measures.

The more I think about validity, the more I think that conventional validation is really a matter of social process. Measures are valid when other people—subjects, peers, reviewers—agree that they are valid. Construct validity is an obvious example. Knight maintains that you have construct validity when you operationalize a measure in a such a way that it is acceptable to other people (p. 136). You are investigating what you set out to investigate because other people agree with your description of the concept. Face validity seems to be similar. Presumably, face validity increases with the number of people who agree that your measures are valid. Content validity, too, depends on some kind of agreement about the capacity of your questions to get at the domain or dimensions of the concept. In each case validation seems to be a matter of fitting your research into a social consensus about the relationship between a measure and a concept. Here's how one researcher put the matter:

[Mishler] starts from the process of validating (instead of from the state of validity) and defines "validation as the social construction of knowledge", by which we "evaluate the 'trustworthiness' of reported observations, interpretations, and generalizations". Finally, "reformulating validation as the social discourse through which trustworthiness is established elides such familiar shibboleths as reliability, falsifiability, and objectivity" (qtd. in Flick, An introduction to qualitative research, pp. 226-227).

So what happens when you use a method or measure in an unconventional way or, more to the point, if you invent a method or measure from scratch? Clearly, you no longer enjoy as much social approval as a more conventional researcher. Does that mean that your findings will be invalid? No. It does mean that your research may have to be the catalyst for a new or revised social consensus. The tricky part is that your findings may be the only things that justify the validity of your research. In other words, you may have to do all of the research in order to prove that your approach was valid. However, you're not alone. I suspect that a lot of qualitative researcher use their findings to justify their design. Hine implied as much at the conclusion of her article in Internet inquiry. She wrote that "the key recourse has to be the dialogues in which the study is able to engage: If we can do studies [. . .] that say something interesting and that advance debate [. . .], then our studies will have some claim to adequacy" (p. 18-19). I can imagine David Gauntlett saying something similar about Serious Play. Knight calls this a "post-structuralist" view of validity (p. 135). But I think that we distort the discussion of research methods when we posit this kind of validation as an exception to the norm. Instead, it is better to think of it as producing the norm. To my mind, the various discourses of validity reflect something like the Kuhnian model of scientific progress, a dialectic between paradigm and disruption.

Karl